2024-09-17T21:35:41.043 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T21:35:41.061 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:35:41.144 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503 branch: squid description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909503' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi no_nested_subset: false openstack: - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: extra_packages: - rbd-nbd flavor: default sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - osd.0 - osd.1 - osd.2 - - mon.b - mgr.x - osd.3 - osd.4 - osd.5 - - client.0 seed: 1983 sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f sleep_before_teardown: 0 subset: 79/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b0d356b98e08f6bc59ef15652f0d0e4298fc83c7 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGX8AOqRTFSCaQ6E/IziRJ+TRam/+XtROAUO3Pjg2w7PcZP9QBOvlFLrOI+PXjgSGKfyfwrx+Gl6lmZfxWcuQu0= smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrxYT9MOB+pNzlpfVQR2VNbiddNqPlw+XYA8wUVZPyE7sS1PBKhSuTQoNuMQNmcUvrzzYvQa5Wvtj4S9aYUpWg= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIS20mlmssYCVZ0CQnBkLNX6IYCWDqBZQm+i6aGk7VbqMxL9f5YLDnqd+aDdyGI6pO1kfygqgA45KdMdlg+t/pg= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 holebdy: 512 nbd: true ops: 6000 punch_holes: true readbdy: 512 truncbdy: 512 writebdy: 512 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_21:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T21:35:41.144 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa; will attempt to use it 2024-09-17T21:35:41.145 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa/tasks 2024-09-17T21:35:41.145 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T21:35:41.146 INFO:teuthology.task.internal:Checking packages... 2024-09-17T21:35:41.170 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c2ddf9721ef254645201cdbabd9c8f49c30a586f' 2024-09-17T21:35:41.171 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T21:35:41.171 INFO:teuthology.packaging:ref: None 2024-09-17T21:35:41.171 INFO:teuthology.packaging:tag: None 2024-09-17T21:35:41.171 INFO:teuthology.packaging:branch: squid 2024-09-17T21:35:41.171 INFO:teuthology.packaging:sha1: c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:35:41.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-17T21:35:41.371 INFO:teuthology.task.internal:Found packages for ceph version 19.2.0-0 2024-09-17T21:35:41.373 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T21:35:41.379 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T21:35:41.379 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T21:35:41.428 INFO:teuthology.task.internal:Saving configuration 2024-09-17T21:35:41.442 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T21:35:41.448 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T21:35:41.473 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 21:30:39.195512', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGX8AOqRTFSCaQ6E/IziRJ+TRam/+XtROAUO3Pjg2w7PcZP9QBOvlFLrOI+PXjgSGKfyfwrx+Gl6lmZfxWcuQu0='} 2024-09-17T21:35:41.494 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 21:30:39.196836', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrxYT9MOB+pNzlpfVQR2VNbiddNqPlw+XYA8wUVZPyE7sS1PBKhSuTQoNuMQNmcUvrzzYvQa5Wvtj4S9aYUpWg='} 2024-09-17T21:35:41.515 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 21:30:39.193960', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIS20mlmssYCVZ0CQnBkLNX6IYCWDqBZQm+i6aGk7VbqMxL9f5YLDnqd+aDdyGI6pO1kfygqgA45KdMdlg+t/pg='} 2024-09-17T21:35:41.515 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T21:35:41.522 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'] 2024-09-17T21:35:41.522 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'] 2024-09-17T21:35:41.522 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['client.0'] 2024-09-17T21:35:41.522 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T21:35:41.599 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdd46c05120>, signals=[15]) 2024-09-17T21:35:41.599 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T21:35:41.605 INFO:teuthology.task.internal:Opening connections... 2024-09-17T21:35:41.605 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:35:41.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:35:41.694 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:35:41.694 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:35:41.773 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-09-17T21:35:41.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:35:41.858 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T21:35:41.865 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-17T21:35:41.883 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-17T21:35:41.883 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-17T21:35:41.941 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T21:35:41.942 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T21:35:41.943 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-17T21:35:41.960 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-17T21:35:41.978 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-17T21:35:41.979 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T21:35:42.036 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-17T21:35:42.037 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T21:35:42.037 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T21:35:42.037 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T21:35:42.037 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-17T21:35:42.051 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-09-17T21:35:42.070 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-09-17T21:35:42.070 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-09-17T21:35:42.127 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-09-17T21:35:42.127 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-09-17T21:35:42.127 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-09-17T21:35:42.127 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T21:35:42.128 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T21:35:42.129 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-09-17T21:35:42.143 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T21:35:42.151 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T21:35:42.157 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T21:35:42.158 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:35:42.161 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:35:42.163 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-09-17T21:35:42.183 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T21:35:42.190 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T21:35:42.190 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:35:42.220 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:35:42.223 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-09-17T21:35:42.256 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T21:35:42.314 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T21:35:42.329 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T21:35:42.329 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'client.0': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T21:35:42.329 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:35:42.330 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:35:42.330 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T21:35:42.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:35:42.348 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-17T21:35:42.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:35:42.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:35:42.351 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-17T21:35:42.352 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-17T21:35:42.406 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-503.el9.x86_64 2024-09-17T21:35:42.407 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.14.0-503.el9.x86_64 2024-09-17T21:35:42.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-09-17T21:35:42.409 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-17T21:35:42.410 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-503.el9.x86_64 2024-09-17T21:35:42.410 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-09-17T21:35:42.412 INFO:teuthology.orchestra.run.smithi165.stdout:5.14.0-503.el9.x86_64 2024-09-17T21:35:42.413 INFO:teuthology.task.kernel:Running kernel on smithi165: 5.14.0-503.el9.x86_64 2024-09-17T21:35:42.413 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum install -y kernel 2024-09-17T21:35:44.782 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-17T21:35:45.089 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.3 MB 00:00 2024-09-17T21:35:45.489 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.3 MB 00:01 2024-09-17T21:35:47.299 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-17T21:35:51.974 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.5 MB 00:00 2024-09-17T21:35:51.990 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 20 MB 00:06 2024-09-17T21:35:52.115 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 3.7 MB/s | 20 MB 00:05 2024-09-17T21:35:54.300 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-17T21:35:55.246 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-17T21:35:56.507 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.5 MB 00:00 2024-09-17T21:35:57.724 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-17T21:35:57.974 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 79 kB/s | 19 kB 00:00 2024-09-17T21:35:58.962 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-17T21:35:59.394 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 19 kB 00:00 2024-09-17T21:36:00.445 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-17T21:36:00.777 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T21:36:02.750 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T21:36:02.750 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T21:36:02.842 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:36:02.848 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 79 M 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 126 M 2024-09-17T21:36:02.849 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:36:02.978 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 332 kB/s | 36 kB 00:00 2024-09-17T21:36:03.353 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-17T21:36:03.729 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 38 MB/s | 29 MB 00:00 2024-09-17T21:36:04.062 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-17T21:36:04.062 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:36:04.062 INFO:teuthology.orchestra.run.smithi007.stdout:Total 65 MB/s | 79 MB 00:01 2024-09-17T21:36:04.359 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-17T21:36:04.766 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:36:05.176 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:36:05.176 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:36:06.068 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T21:36:06.166 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:36:06.166 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:36:06.249 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T21:36:06.249 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T21:36:06.348 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:36:06.352 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T21:36:06.353 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout:Install 4 Packages 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 79 M 2024-09-17T21:36:06.354 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 126 M 2024-09-17T21:36:06.355 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:36:06.527 INFO:teuthology.orchestra.run.smithi165.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 287 kB/s | 36 kB 00:00 2024-09-17T21:36:08.116 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T21:36:08.117 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T21:36:08.216 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T21:36:08.221 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:36:08.222 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:36:08.222 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:36:08.222 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-17T21:36:08.222 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:36:08.224 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 79 M 2024-09-17T21:36:08.224 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 126 M 2024-09-17T21:36:08.224 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:36:08.750 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:36:08.777 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 119 kB/s | 36 kB 00:00 2024-09-17T21:36:08.952 INFO:teuthology.orchestra.run.smithi165.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-09-17T21:36:09.425 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:36:09.514 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:09.903 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-17T21:36:10.561 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 16 MB/s | 29 MB 00:01 2024-09-17T21:36:10.887 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 14 MB/s | 35 MB 00:02 2024-09-17T21:36:10.887 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:36:10.887 INFO:teuthology.orchestra.run.smithi053.stdout:Total 30 MB/s | 79 MB 00:02 2024-09-17T21:36:10.927 INFO:teuthology.orchestra.run.smithi165.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 7.6 MB/s | 35 MB 00:04 2024-09-17T21:36:11.444 INFO:teuthology.orchestra.run.smithi165.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 5.8 MB/s | 29 MB 00:04 2024-09-17T21:36:11.444 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:36:11.444 INFO:teuthology.orchestra.run.smithi165.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-17T21:36:11.600 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:36:12.000 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:36:12.001 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:36:12.124 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:36:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:12.433 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:12.470 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:36:12.471 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:36:12.981 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:36:12.982 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:36:13.441 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:36:13.441 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:36:15.528 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:36:15.923 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:36:16.180 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:36:16.262 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:16.560 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:36:16.645 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:17.554 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:17.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:36:18.999 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:19.142 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:19.409 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:36:19.546 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:22.743 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:36:24.264 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:24.309 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:36:24.501 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:36:24.545 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:36:29.393 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:36:29.454 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:02.040 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:37:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:37:02.410 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.295 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.296 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.296 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:37:03.296 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:37:03.689 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:03.779 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.779 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:03.779 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T21:37:03.779 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:37:03.780 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:37:03.780 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T21:37:03.780 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:37:03.780 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:03.839 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:03.840 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:03.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-17T21:37:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-17T21:37:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-17T21:37:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-17T21:37:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-17T21:37:03.872 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-17T21:37:03.873 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:37:03.874 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T21:37:03.875 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T21:37:03.875 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-508.el9.x86_64 2024-09-17T21:37:03.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T21:37:04.580 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-17T21:37:05.899 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T21:37:05.917 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-17T21:37:05.920 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-17T21:37:05.936 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.6PwyncgLRz 2024-09-17T21:37:05.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6PwyncgLRz 2024-09-17T21:37:06.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.6PwyncgLRz 2024-09-17T21:37:06.146 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.6PwyncgLRz is 6KB 2024-09-17T21:37:06.194 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.6PwyncgLRz 2024-09-17T21:37:06.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T21:37:06.272 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T21:37:06.272 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T21:37:06.272 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T21:37:06.272 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T21:37:06.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T21:37:06.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-17T21:37:07.666 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:07.706 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:08.081 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:08.081 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:37:08.081 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:37:08.081 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:37:08.839 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:08.886 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:08.941 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:37:08.942 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:37:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T21:37:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T21:37:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T21:37:09.297 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:09.378 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:09.379 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:09.379 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T21:37:09.379 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:37:09.379 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:37:09.379 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-17T21:37:09.379 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi165.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:37:09.379 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:09.438 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:09.438 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:09.439 DEBUG:teuthology.orchestra.run.smithi165:> sudo rpm -qi grub2-tools 2024-09-17T21:37:09.471 INFO:teuthology.orchestra.run.smithi165.stdout:Name : grub2-tools 2024-09-17T21:37:09.471 INFO:teuthology.orchestra.run.smithi165.stdout:Epoch : 1 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Version : 2.06 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Release : 82.el9 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: x86_64 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Group : Unspecified 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Size : 8274670 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:License : GPLv3+ 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T21:37:09.472 INFO:teuthology.orchestra.run.smithi165.stdout:Packager : builder@centos.org 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:Vendor : CentOS 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:Summary : Support tools for GRUB. 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:Description : 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T21:37:09.473 INFO:teuthology.orchestra.run.smithi165.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T21:37:09.474 INFO:teuthology.orchestra.run.smithi165.stdout:hardware devices. 2024-09-17T21:37:09.474 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:37:09.474 INFO:teuthology.orchestra.run.smithi165.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T21:37:09.474 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T21:37:09.475 INFO:teuthology.task.kernel:Updating grub on smithi165 to boot 5.14.0-508.el9.x86_64 2024-09-17T21:37:09.475 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T21:37:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:37:10.105 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:37:10.178 INFO:teuthology.orchestra.run.smithi165.stderr:Generating grub configuration file ... 2024-09-17T21:37:10.449 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:10.538 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.538 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:10.538 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T21:37:10.539 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T21:37:10.539 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T21:37:10.539 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T21:37:10.539 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T21:37:10.539 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T21:37:10.597 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T21:37:10.598 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T21:37:10.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-09-17T21:37:10.629 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-09-17T21:37:10.629 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-09-17T21:37:10.629 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-09-17T21:37:10.629 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 82.el9 2024-09-17T21:37:10.629 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8274670 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-09-17T21:37:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:37:10.631 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T21:37:10.632 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T21:37:10.632 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 5.14.0-508.el9.x86_64 2024-09-17T21:37:10.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T21:37:11.221 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-09-17T21:37:11.603 INFO:teuthology.orchestra.run.smithi165.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T21:37:11.628 INFO:teuthology.orchestra.run.smithi165.stderr:done 2024-09-17T21:37:11.630 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-09-17T21:37:11.686 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.GgGTh6i1PD 2024-09-17T21:37:11.687 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GgGTh6i1PD 2024-09-17T21:37:11.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.GgGTh6i1PD 2024-09-17T21:37:11.889 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.GgGTh6i1PD is 6KB 2024-09-17T21:37:11.937 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.GgGTh6i1PD 2024-09-17T21:37:11.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T21:37:12.015 INFO:teuthology.orchestra.run.smithi165.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T21:37:12.015 INFO:teuthology.orchestra.run.smithi165.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T21:37:12.016 INFO:teuthology.orchestra.run.smithi165.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T21:37:12.016 INFO:teuthology.orchestra.run.smithi165.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T21:37:12.017 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T21:37:12.159 DEBUG:teuthology.orchestra.run.smithi165:> sudo shutdown -r now 2024-09-17T21:37:12.580 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T21:37:12.624 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-09-17T21:37:12.626 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-17T21:37:12.641 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.FK4JqW9v3O 2024-09-17T21:37:12.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FK4JqW9v3O 2024-09-17T21:37:12.710 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.FK4JqW9v3O 2024-09-17T21:37:12.852 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.FK4JqW9v3O is 6KB 2024-09-17T21:37:12.902 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.FK4JqW9v3O 2024-09-17T21:37:12.916 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T21:37:12.979 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T21:37:12.979 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T21:37:12.980 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T21:37:12.980 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T21:37:12.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T21:37:13.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-09-17T21:37:36.457 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T21:37:36.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T21:37:36.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:37:42.165 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-17T21:37:42.165 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-17T21:37:42.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:37:43.133 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T21:37:43.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-17T21:37:43.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:38:36.514 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T21:38:42.173 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T21:38:43.137 DEBUG:teuthology.orchestra.remote:timed out 2024-09-17T21:38:45.517 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T21:38:45.518 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:38:46.237 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-17T21:38:46.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T21:38:46.625 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T21:38:46.626 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-17T21:38:46.641 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-508.el9.x86_64 2024-09-17T21:38:46.642 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T21:38:46.642 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:38:46.642 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T21:38:47.642 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T21:38:47.643 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:38:47.742 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-09-17T21:38:47.766 DEBUG:teuthology.parallel:result is None 2024-09-17T21:38:51.174 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-17T21:38:51.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:38:52.138 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-17T21:38:52.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:38:52.647 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-17T21:38:53.030 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-17T21:38:53.030 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T21:38:53.031 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-17T21:38:53.047 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-508.el9.x86_64 2024-09-17T21:38:53.047 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T21:38:53.047 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:38:53.047 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T21:38:54.048 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T21:38:54.049 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:38:54.122 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-09-17T21:38:54.145 DEBUG:teuthology.parallel:result is None 2024-09-17T21:38:58.602 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-17T21:38:58.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-17T21:38:58.945 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-17T21:38:58.945 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-09-17T21:38:58.959 INFO:teuthology.orchestra.run.smithi165.stdout:5.14.0-508.el9.x86_64 2024-09-17T21:38:58.960 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T21:38:58.960 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T21:38:58.960 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T21:38:59.960 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2024-09-17T21:38:59.960 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T21:39:00.041 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-09-17T21:39:00.064 DEBUG:teuthology.parallel:result is None 2024-09-17T21:39:00.064 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T21:39:00.074 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T21:39:00.074 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:39:00.077 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:39:00.079 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T21:39:00.096 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T21:39:00.122 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T21:39:00.137 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T21:39:00.138 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:39:00.141 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:39:00.143 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T21:39:00.188 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T21:39:00.195 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T21:39:00.195 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:39:00.197 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:39:00.215 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T21:39:00.228 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.240 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.240 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.251 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.252 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.263 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T21:39:00.265 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T21:39:00.271 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T21:39:00.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:39:00.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:39:00.308 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T21:39:00.335 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T21:39:00.352 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T21:39:00.352 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:39:00.356 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:39:00.378 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T21:39:00.394 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:00.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:00.507 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:00.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:00.631 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:39:00.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:39:00.697 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:00.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:00.796 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:00.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:00.925 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:39:00.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:39:00.993 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:01.032 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T21:39:01.087 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:01.168 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T21:39:01.208 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:39:01.208 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T21:39:01.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-17T21:39:01.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-17T21:39:01.278 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-09-17T21:39:01.322 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:39:01.334 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:39:01.399 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:39:01.843 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T21:39:01.851 INFO:teuthology.task.internal:Starting timer... 2024-09-17T21:39:01.851 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T21:39:01.883 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T21:39:01.930 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-17T21:39:01.930 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-17T21:39:01.989 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-17T21:39:01.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-17T21:39:02.032 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-17T21:39:02.034 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-09-17T21:39:02.137 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-09-17T21:39:02.140 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T21:39:02.141 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-17T21:39:02.173 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-17T21:39:02.174 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-17T21:39:02.223 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-17T21:39:02.223 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-09-17T21:39:02.254 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-09-17T21:39:02.254 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-09-17T21:39:02.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:39:02.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:39:02.286 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:39:02.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:39:02.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:39:02.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:39:02.392 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T21:39:02.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-17T21:39:02.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-17T21:39:02.467 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-09-17T21:39:02.560 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T21:39:02.569 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-17T21:39:02.572 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-17T21:39:02.681 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T21:39:02.693 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T21:39:02.694 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi053.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T21:42:50.517 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-09-17T21:42:50.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T21:42:50.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:42:50.598 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-17T21:42:50.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-17T21:42:50.673 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-17T21:42:50.674 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:42:50.749 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-17T21:42:50.833 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-17T21:42:50.834 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-17T21:42:50.835 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T21:42:50.909 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-09-17T21:42:50.988 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-09-17T21:42:50.989 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T21:42:51.000 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T21:42:51.000 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:42:51.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:42:51.003 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:42:51.003 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:42:51.006 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T21:42:51.006 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:42:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T21:42:51.038 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T21:42:51.050 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T21:42:51.055 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T21:42:51.057 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T21:42:51.071 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T21:42:51.077 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-17T21:42:51.083 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-17T21:42:51.089 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-17T21:42:51.095 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-17T21:42:51.095 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-09-17T21:42:51.104 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T21:42:51.106 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-09-17T21:42:51.111 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T21:42:51.119 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T21:42:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T21:42:51.127 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T21:42:51.135 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T21:42:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:42:51.185 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:42:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-17T21:42:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.190 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.199 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:42:51.202 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:42:51.202 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-17T21:42:51.202 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.203 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.203 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.203 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:42:51.204 INFO:teuthology.run_tasks:Running task install... 2024-09-17T21:42:51.212 DEBUG:teuthology.task.install:project ceph 2024-09-17T21:42:51.212 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}} 2024-09-17T21:42:51.212 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'} 2024-09-17T21:42:51.212 INFO:teuthology.task.install:Using flavor: default 2024-09-17T21:42:51.218 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T21:42:51.218 INFO:teuthology.task.install:extra packages: ['rbd-nbd'] 2024-09-17T21:42:51.218 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T21:42:51.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:42:51.220 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T21:42:51.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:42:51.221 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': ['rbd-nbd'], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'tag': None, 'wait_for_package': False} 2024-09-17T21:42:51.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:42:51.365 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T21:42:51.365 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T21:42:51.375 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T21:42:51.375 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T21:42:51.377 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/ 2024-09-17T21:42:51.378 INFO:teuthology.task.install.rpm:Package version is 19.1.1-330.gc2ddf972 2024-09-17T21:42:51.479 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T21:42:51.479 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:42:51.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T21:42:51.486 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T21:42:51.486 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:42:51.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T21:42:51.491 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/c2ddf9721ef254645201cdbabd9c8f49c30a586f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T21:42:51.492 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:42:51.492 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T21:42:51.506 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T21:42:51.506 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T21:42:51.514 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T21:42:51.514 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T21:42:51.518 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, rbd-nbd on remote rpm x86_64 2024-09-17T21:42:51.518 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c2ddf9721ef254645201cdbabd9c8f49c30a586f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T21:42:51.573 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T21:42:51.582 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T21:42:51.584 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T21:42:51.649 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T21:42:51.659 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T21:42:51.660 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T21:42:51.714 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-17T21:42:51.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-17T21:42:51.723 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-09-17T21:42:51.725 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-09-17T21:42:51.726 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-17T21:42:51.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-17T21:42:51.956 INFO:teuthology.orchestra.run.smithi165.stdout:45 files removed 2024-09-17T21:42:51.960 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-17T21:42:51.971 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-17T21:42:51.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw 2024-09-17T21:42:51.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-09-17T21:42:51.999 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw 2024-09-17T21:42:52.532 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 317 kB/s | 84 kB 00:00 2024-09-17T21:42:52.557 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 313 kB/s | 84 kB 00:00 2024-09-17T21:42:52.560 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-09-17T21:42:52.824 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-17T21:42:52.844 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-17T21:42:52.849 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-17T21:42:53.108 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-17T21:42:53.125 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-17T21:42:53.236 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-09-17T21:42:53.875 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-17T21:42:54.041 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 9.2 MB/s | 8.3 MB 00:00 2024-09-17T21:42:54.511 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.3 MB 00:01 2024-09-17T21:42:55.785 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-17T21:42:56.051 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-17T21:42:56.512 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-17T21:43:00.726 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 8.0 MB/s | 6.5 MB 00:00 2024-09-17T21:43:01.443 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.5 MB 00:01 2024-09-17T21:43:01.854 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-17T21:43:02.552 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-17T21:43:02.903 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 136 kB/s | 19 kB 00:00 2024-09-17T21:43:03.807 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-17T21:43:03.855 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-17T21:43:03.965 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-17T21:43:04.893 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-17T21:43:09.279 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T21:43:09.637 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-17T21:43:10.482 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-17T21:43:11.506 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:43:11.515 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T21:43:11.516 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T21:43:11.517 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:11.518 INFO:teuthology.orchestra.run.smithi165.stdout:Install 30 Packages 2024-09-17T21:43:11.519 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2024-09-17T21:43:11.519 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:11.519 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 60 M 2024-09-17T21:43:11.519 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:43:11.942 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T21:43:11.951 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T21:43:11.952 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T21:43:11.953 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout:Install 30 Packages 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-09-17T21:43:11.954 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:11.955 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 60 M 2024-09-17T21:43:11.955 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:43:12.147 INFO:teuthology.orchestra.run.smithi165.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 9.4 MB/s | 5.4 MB 00:00 2024-09-17T21:43:12.189 INFO:teuthology.orchestra.run.smithi165.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 608 kB/s | 25 kB 00:00 2024-09-17T21:43:12.298 INFO:teuthology.orchestra.run.smithi165.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 6.8 MB/s | 751 kB 00:00 2024-09-17T21:43:12.423 INFO:teuthology.orchestra.run.smithi165.stdout:(4/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 12 MB/s | 11 MB 00:00 2024-09-17T21:43:12.457 INFO:teuthology.orchestra.run.smithi165.stdout:(5/32): libradosstriper1-19.1.1-330.gc2ddf972.e 3.1 MB/s | 506 kB 00:00 2024-09-17T21:43:12.515 INFO:teuthology.orchestra.run.smithi165.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 774 kB/s | 45 kB 00:00 2024-09-17T21:43:12.774 INFO:teuthology.orchestra.run.smithi165.stdout:(7/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 18 MB/s | 21 MB 00:01 2024-09-17T21:43:12.785 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 11 M 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-17T21:43:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.4 M 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.2 M 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.4 M 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 21 M 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 25 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 751 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T21:43:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 506 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 5.3 M 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 45 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 136 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 162 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 320 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 300 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 99 k 2024-09-17T21:43:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout:Installing weak dependencies: 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout:Install 30 Packages 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 2 Packages 2024-09-17T21:43:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:12.803 INFO:teuthology.orchestra.run.smithi165.stdout:(8/32): python3-ceph-common-19.1.1-330.gc2ddf97 480 kB/s | 136 kB 00:00 2024-09-17T21:43:12.803 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 60 M 2024-09-17T21:43:12.803 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:43:12.825 INFO:teuthology.orchestra.run.smithi165.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 3.2 MB/s | 162 kB 00:00 2024-09-17T21:43:12.850 INFO:teuthology.orchestra.run.smithi165.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.2 MB/s | 320 kB 00:00 2024-09-17T21:43:12.883 INFO:teuthology.orchestra.run.smithi165.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-17T21:43:12.909 INFO:teuthology.orchestra.run.smithi165.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-17T21:43:12.984 INFO:teuthology.orchestra.run.smithi165.stdout:(13/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 9.5 MB/s | 5.3 MB 00:00 2024-09-17T21:43:13.043 INFO:teuthology.orchestra.run.smithi165.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 671 kB/s | 107 kB 00:00 2024-09-17T21:43:13.077 INFO:teuthology.orchestra.run.smithi165.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-09-17T21:43:13.127 INFO:teuthology.orchestra.run.smithi165.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 736 kB/s | 160 kB 00:00 2024-09-17T21:43:13.152 INFO:teuthology.orchestra.run.smithi165.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-09-17T21:43:13.178 INFO:teuthology.orchestra.run.smithi165.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-09-17T21:43:13.203 INFO:teuthology.orchestra.run.smithi165.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-17T21:43:13.222 INFO:teuthology.orchestra.run.smithi007.stdout:(1/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 8.9 MB/s | 11 MB 00:01 2024-09-17T21:43:13.228 INFO:teuthology.orchestra.run.smithi165.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-09-17T21:43:13.253 INFO:teuthology.orchestra.run.smithi165.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-17T21:43:13.279 INFO:teuthology.orchestra.run.smithi165.stdout:(22/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-09-17T21:43:13.289 INFO:teuthology.orchestra.run.smithi007.stdout:(2/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 4.3 MB/s | 5.4 MB 00:01 2024-09-17T21:43:13.314 INFO:teuthology.orchestra.run.smithi007.stdout:(3/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 274 kB/s | 25 kB 00:00 2024-09-17T21:43:13.338 INFO:teuthology.orchestra.run.smithi165.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-17T21:43:13.363 INFO:teuthology.orchestra.run.smithi165.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-17T21:43:13.381 INFO:teuthology.orchestra.run.smithi007.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 7.4 MB/s | 506 kB 00:00 2024-09-17T21:43:13.388 INFO:teuthology.orchestra.run.smithi165.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 277 kB/s | 26 kB 00:00 2024-09-17T21:43:13.406 INFO:teuthology.orchestra.run.smithi007.stdout:(5/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 6.3 MB/s | 751 kB 00:00 2024-09-17T21:43:13.413 INFO:teuthology.orchestra.run.smithi165.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T21:43:13.447 INFO:teuthology.orchestra.run.smithi165.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T21:43:13.465 INFO:teuthology.orchestra.run.smithi007.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 773 kB/s | 45 kB 00:00 2024-09-17T21:43:13.497 INFO:teuthology.orchestra.run.smithi165.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-17T21:43:13.532 INFO:teuthology.orchestra.run.smithi007.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.0 MB/s | 136 kB 00:00 2024-09-17T21:43:13.548 INFO:teuthology.orchestra.run.smithi165.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-17T21:43:13.656 INFO:teuthology.orchestra.run.smithi165.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-17T21:43:13.757 INFO:teuthology.orchestra.run.smithi007.stdout:(8/32): ceph-common-19.1.1-330.gc2ddf972.el9.x8 12 MB/s | 21 MB 00:01 2024-09-17T21:43:13.773 INFO:teuthology.orchestra.run.smithi165.stdout:(31/32): librados2-19.1.1-330.gc2ddf972.el9.x86 12 MB/s | 3.4 MB 00:00 2024-09-17T21:43:13.783 INFO:teuthology.orchestra.run.smithi007.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 646 kB/s | 162 kB 00:00 2024-09-17T21:43:13.824 INFO:teuthology.orchestra.run.smithi007.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 4.7 MB/s | 320 kB 00:00 2024-09-17T21:43:13.832 INFO:teuthology.orchestra.run.smithi165.stdout:(32/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-17T21:43:13.837 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:13.837 INFO:teuthology.orchestra.run.smithi165.stdout:Total 26 MB/s | 60 MB 00:02 2024-09-17T21:43:13.850 INFO:teuthology.orchestra.run.smithi007.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 4.4 MB/s | 300 kB 00:00 2024-09-17T21:43:13.875 INFO:teuthology.orchestra.run.smithi007.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T21:43:13.941 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:43:13.950 INFO:teuthology.orchestra.run.smithi007.stdout:(13/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_6 9.4 MB/s | 5.3 MB 00:00 2024-09-17T21:43:13.975 INFO:teuthology.orchestra.run.smithi053.stdout:(1/32): ceph-base-19.1.1-330.gc2ddf972.el9.x86_ 5.1 MB/s | 5.4 MB 00:01 2024-09-17T21:43:13.976 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:43:13.976 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:43:14.017 INFO:teuthology.orchestra.run.smithi053.stdout:(2/32): ceph-selinux-19.1.1-330.gc2ddf972.el9.x 607 kB/s | 25 kB 00:00 2024-09-17T21:43:14.092 INFO:teuthology.orchestra.run.smithi007.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-09-17T21:43:14.109 INFO:teuthology.orchestra.run.smithi053.stdout:(3/32): libcephfs2-19.1.1-330.gc2ddf972.el9.x86 8.0 MB/s | 751 kB 00:00 2024-09-17T21:43:14.118 INFO:teuthology.orchestra.run.smithi007.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 399 kB/s | 107 kB 00:00 2024-09-17T21:43:14.143 INFO:teuthology.orchestra.run.smithi007.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 599 kB/s | 160 kB 00:00 2024-09-17T21:43:14.184 INFO:teuthology.orchestra.run.smithi053.stdout:(4/32): libradosstriper1-19.1.1-330.gc2ddf972.e 6.6 MB/s | 506 kB 00:00 2024-09-17T21:43:14.193 INFO:teuthology.orchestra.run.smithi007.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 845 kB/s | 42 kB 00:00 2024-09-17T21:43:14.227 INFO:teuthology.orchestra.run.smithi007.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-17T21:43:14.277 INFO:teuthology.orchestra.run.smithi007.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-09-17T21:43:14.302 INFO:teuthology.orchestra.run.smithi007.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-17T21:43:14.328 INFO:teuthology.orchestra.run.smithi007.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 357 kB/s | 36 kB 00:00 2024-09-17T21:43:14.345 INFO:teuthology.orchestra.run.smithi007.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 705 kB/s | 47 kB 00:00 2024-09-17T21:43:14.403 INFO:teuthology.orchestra.run.smithi007.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 446 kB/s | 26 kB 00:00 2024-09-17T21:43:14.417 INFO:teuthology.orchestra.run.smithi053.stdout:(5/32): ceph-radosgw-19.1.1-330.gc2ddf972.el9.x 7.0 MB/s | 11 MB 00:01 2024-09-17T21:43:14.420 INFO:teuthology.orchestra.run.smithi007.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-17T21:43:14.445 INFO:teuthology.orchestra.run.smithi007.stdout:(25/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-17T21:43:14.468 INFO:teuthology.orchestra.run.smithi053.stdout:(6/32): python3-ceph-argparse-19.1.1-330.gc2ddf 906 kB/s | 45 kB 00:00 2024-09-17T21:43:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T21:43:14.504 INFO:teuthology.orchestra.run.smithi007.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T21:43:14.526 INFO:teuthology.orchestra.run.smithi053.stdout:(7/32): python3-ceph-common-19.1.1-330.gc2ddf97 2.3 MB/s | 136 kB 00:00 2024-09-17T21:43:14.538 INFO:teuthology.orchestra.run.smithi007.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-17T21:43:14.635 INFO:teuthology.orchestra.run.smithi053.stdout:(8/32): librgw2-19.1.1-330.gc2ddf972.el9.x86_64 12 MB/s | 5.3 MB 00:00 2024-09-17T21:43:14.638 INFO:teuthology.orchestra.run.smithi007.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-17T21:43:14.646 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:43:14.646 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:43:14.660 INFO:teuthology.orchestra.run.smithi053.stdout:(9/32): python3-cephfs-19.1.1-330.gc2ddf972.el9 1.2 MB/s | 162 kB 00:00 2024-09-17T21:43:14.685 INFO:teuthology.orchestra.run.smithi053.stdout:(10/32): python3-rados-19.1.1-330.gc2ddf972.el9 6.3 MB/s | 320 kB 00:00 2024-09-17T21:43:14.705 INFO:teuthology.orchestra.run.smithi007.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-17T21:43:14.711 INFO:teuthology.orchestra.run.smithi053.stdout:(11/32): python3-rbd-19.1.1-330.gc2ddf972.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-17T21:43:14.736 INFO:teuthology.orchestra.run.smithi053.stdout:(12/32): python3-rgw-19.1.1-330.gc2ddf972.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-17T21:43:14.881 INFO:teuthology.orchestra.run.smithi007.stdout:(31/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 13 MB/s | 3.2 MB 00:00 2024-09-17T21:43:14.956 INFO:teuthology.orchestra.run.smithi007.stdout:(32/32): librados2-19.1.1-330.gc2ddf972.el9.x86 8.2 MB/s | 3.4 MB 00:00 2024-09-17T21:43:14.961 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:14.962 INFO:teuthology.orchestra.run.smithi053.stdout:(13/32): ceph-common-19.1.1-330.gc2ddf972.el9.x 10 MB/s | 21 MB 00:02 2024-09-17T21:43:14.962 INFO:teuthology.orchestra.run.smithi007.stdout:Total 20 MB/s | 60 MB 00:03 2024-09-17T21:43:15.020 INFO:teuthology.orchestra.run.smithi053.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-17T21:43:15.069 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:43:15.111 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:43:15.111 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:43:15.120 INFO:teuthology.orchestra.run.smithi053.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 417 kB/s | 160 kB 00:00 2024-09-17T21:43:15.146 INFO:teuthology.orchestra.run.smithi053.stdout:(16/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 246 kB/s | 45 kB 00:00 2024-09-17T21:43:15.179 INFO:teuthology.orchestra.run.smithi053.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-17T21:43:15.204 INFO:teuthology.orchestra.run.smithi053.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-09-17T21:43:15.230 INFO:teuthology.orchestra.run.smithi053.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-17T21:43:15.255 INFO:teuthology.orchestra.run.smithi053.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-09-17T21:43:15.280 INFO:teuthology.orchestra.run.smithi053.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 475 kB/s | 36 kB 00:00 2024-09-17T21:43:15.306 INFO:teuthology.orchestra.run.smithi053.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-17T21:43:15.364 INFO:teuthology.orchestra.run.smithi053.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-17T21:43:15.398 INFO:teuthology.orchestra.run.smithi053.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2024-09-17T21:43:15.423 INFO:teuthology.orchestra.run.smithi053.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 257 kB/s | 26 kB 00:00 2024-09-17T21:43:15.448 INFO:teuthology.orchestra.run.smithi053.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-17T21:43:15.482 INFO:teuthology.orchestra.run.smithi053.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-17T21:43:15.515 INFO:teuthology.orchestra.run.smithi053.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-17T21:43:15.599 INFO:teuthology.orchestra.run.smithi053.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-17T21:43:15.716 INFO:teuthology.orchestra.run.smithi053.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-17T21:43:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:43:15.804 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:43:15.833 INFO:teuthology.orchestra.run.smithi053.stdout:(31/32): librados2-19.1.1-330.gc2ddf972.el9.x86 11 MB/s | 3.4 MB 00:00 2024-09-17T21:43:15.900 INFO:teuthology.orchestra.run.smithi053.stdout:(32/32): librbd1-19.1.1-330.gc2ddf972.el9.x86_6 11 MB/s | 3.2 MB 00:00 2024-09-17T21:43:15.905 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:15.905 INFO:teuthology.orchestra.run.smithi053.stdout:Total 19 MB/s | 60 MB 00:03 2024-09-17T21:43:16.014 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:43:16.054 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:43:16.055 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:43:16.127 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:43:16.300 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T21:43:16.329 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T21:43:16.354 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T21:43:16.377 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:16.397 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:16.588 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:16.650 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:16.720 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:43:16.721 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:43:16.989 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T21:43:17.089 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:17.182 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:17.272 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T21:43:17.302 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:17.340 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:43:17.371 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:17.408 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T21:43:17.435 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T21:43:17.452 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T21:43:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T21:43:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T21:43:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:17.522 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:17.543 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:17.552 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:17.624 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:17.659 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T21:43:17.728 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:17.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T21:43:17.790 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:17.801 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T21:43:17.842 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T21:43:17.883 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T21:43:17.912 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T21:43:17.949 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T21:43:18.218 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T21:43:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:43:18.254 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:18.260 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T21:43:18.300 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:18.310 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-17T21:43:18.338 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-17T21:43:18.356 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:18.363 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-17T21:43:18.386 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:18.397 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:18.412 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-17T21:43:18.432 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T21:43:18.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T21:43:18.465 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T21:43:18.472 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:18.536 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:18.564 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T21:43:18.600 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T21:43:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:18.667 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-17T21:43:18.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:18.764 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:18.798 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T21:43:18.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T21:43:18.941 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T21:43:18.948 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T21:43:18.978 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:18.992 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T21:43:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T21:43:19.049 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:19.052 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T21:43:19.072 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-17T21:43:19.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T21:43:19.123 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:19.134 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T21:43:19.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T21:43:19.233 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:19.323 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 8/34 2024-09-17T21:43:19.350 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:19.412 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T21:43:19.429 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 9/34 2024-09-17T21:43:19.456 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:19.466 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-17T21:43:19.510 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 11/34 2024-09-17T21:43:19.547 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:19.577 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T21:43:19.588 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:19.613 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T21:43:19.618 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:19.682 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:19.717 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-17T21:43:19.844 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-17T21:43:19.884 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-17T21:43:19.931 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-17T21:43:19.966 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-17T21:43:19.995 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-17T21:43:20.029 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-17T21:43:20.121 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T21:43:20.150 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:20.230 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T21:43:20.335 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T21:43:20.355 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-17T21:43:20.399 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:20.506 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 22/34 2024-09-17T21:43:20.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 23/34 2024-09-17T21:43:20.582 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-17T21:43:21.089 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-17T21:43:21.118 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:21.181 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 26/34 2024-09-17T21:43:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 27/34 2024-09-17T21:43:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 28/34 2024-09-17T21:43:21.484 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:21.519 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:22.080 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:22.122 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:22.212 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:22.212 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T21:43:22.212 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T21:43:22.212 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:22.248 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:23.103 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:23.148 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:23.744 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:23.744 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:23.785 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:23.791 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:23.903 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:23.903 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T21:43:23.903 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T21:43:23.903 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:23.945 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:24.386 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 29/34 2024-09-17T21:43:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:24.510 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 30/34 2024-09-17T21:43:24.510 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T21:43:24.510 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T21:43:24.510 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:24.537 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:32.404 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-09-17T21:43:32.405 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:33.396 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:33.423 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:33.423 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:33.423 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T21:43:33.423 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:33.423 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:33.424 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:33.425 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:33.507 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:33.508 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-17T21:43:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T21:43:34.699 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T21:43:34.700 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T21:43:34.701 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T21:43:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T21:43:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T21:43:34.702 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-17T21:43:35.023 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-17T21:43:35.024 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-17T21:43:35.024 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:35.551 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:35.551 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T21:43:35.551 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:35.551 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:35.696 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:35.696 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T21:43:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:35.699 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:43:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 32/34 2024-09-17T21:43:36.054 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:36.054 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T21:43:36.054 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:36.054 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T21:43:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:36.089 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-test 2024-09-17T21:43:36.146 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-17T21:43:36.146 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:36.672 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:27 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:43:36.815 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:43:36.815 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:36.815 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:43:36.815 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:36.815 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout:Install 4 Packages 2024-09-17T21:43:36.816 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:36.817 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 49 M 2024-09-17T21:43:36.817 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 209 M 2024-09-17T21:43:36.817 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:43:36.920 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:36.920 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T21:43:36.921 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T21:43:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T21:43:36.923 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T21:43:36.923 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T21:43:36.924 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T21:43:36.997 INFO:teuthology.orchestra.run.smithi165.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-17T21:43:37.038 INFO:teuthology.orchestra.run.smithi165.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T21:43:37.105 INFO:teuthology.orchestra.run.smithi165.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 663 kB/s | 163 kB 00:00 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 1/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 3/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 4/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 5/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 6/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 7/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x 8/34 2024-09-17T21:43:37.340 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86 9/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 10/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 11/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 12/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 13/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-17T21:43:37.341 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-17T21:43:37.342 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-17T21:43:37.343 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-17T21:43:37.343 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-17T21:43:37.343 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 31/34 2024-09-17T21:43:37.343 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-17T21:43:37.343 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 33/34 2024-09-17T21:43:38.048 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:38.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:38.048 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T21:43:38.049 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.050 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:43:38.372 INFO:teuthology.orchestra.run.smithi165.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-17T21:43:38.374 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:38.375 INFO:teuthology.orchestra.run.smithi165.stdout:Total 32 MB/s | 49 MB 00:01 2024-09-17T21:43:38.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-09-17T21:43:38.402 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:43:38.403 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-17T21:43:38.403 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:38.403 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T21:43:38.404 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.405 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:38.406 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:43:38.423 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:43:38.423 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:43:38.731 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-test 2024-09-17T21:43:38.810 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:43:38.810 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:43:38.981 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:43:39.126 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T21:43:39.127 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-09-17T21:43:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:39.129 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 49 M 2024-09-17T21:43:39.129 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 209 M 2024-09-17T21:43:39.129 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:43:39.336 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:29 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:43:39.390 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 832 kB/s | 163 kB 00:00 2024-09-17T21:43:39.391 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:43:39.416 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T21:43:39.463 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T21:43:39.473 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:43:39.473 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 49 M 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 163 k 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:39.474 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 49 M 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 209 M 2024-09-17T21:43:39.475 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:43:39.491 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:39.642 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-17T21:43:39.684 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-17T21:43:39.867 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): libcephsqlite-19.1.1-330.gc2ddf972.el9.x 442 kB/s | 163 kB 00:00 2024-09-17T21:43:39.966 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 386 kB/s | 304 kB 00:00 2024-09-17T21:43:39.991 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 103 kB/s | 64 kB 00:00 2024-09-17T21:43:40.708 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 32 MB/s | 49 MB 00:01 2024-09-17T21:43:40.712 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:40.712 INFO:teuthology.orchestra.run.smithi007.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-17T21:43:40.739 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:43:40.763 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:43:41.167 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:43:41.167 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:43:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:43:41.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T21:43:41.809 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): ceph-test-19.1.1-330.gc2ddf972.el9.x86_6 21 MB/s | 49 MB 00:02 2024-09-17T21:43:41.811 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:41.812 INFO:teuthology.orchestra.run.smithi053.stdout:Total 21 MB/s | 49 MB 00:02 2024-09-17T21:43:41.840 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:43:41.841 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T21:43:41.862 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:43:41.862 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:43:41.868 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:42.271 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:43:42.271 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:43:42.823 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:43:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-17T21:43:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-17T21:43:42.969 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:44.121 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:44.215 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:46.625 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:46.625 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T21:43:46.625 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T21:43:46.625 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T21:43:46.782 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:46.870 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T21:43:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:46.981 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:43:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:46.981 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T21:43:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T21:43:46.982 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:46.982 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:43:47.302 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph 2024-09-17T21:43:47.859 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 3/4 2024-09-17T21:43:47.900 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:38 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:43:47.953 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:48.044 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T21:43:48.047 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T21:43:48.048 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T21:43:48.049 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout:Install 37 Packages 2024-09-17T21:43:48.050 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:48.051 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 30 M 2024-09-17T21:43:48.051 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 108 M 2024-09-17T21:43:48.051 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:43:48.546 INFO:teuthology.orchestra.run.smithi165.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 48 kB/s | 6.5 kB 00:00 2024-09-17T21:43:48.712 INFO:teuthology.orchestra.run.smithi165.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-17T21:43:48.780 INFO:teuthology.orchestra.run.smithi165.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.5 MB/s | 2.4 MB 00:00 2024-09-17T21:43:48.855 INFO:teuthology.orchestra.run.smithi165.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 3.3 MB/s | 249 kB 00:00 2024-09-17T21:43:49.113 INFO:teuthology.orchestra.run.smithi165.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 8.3 MB/s | 4.7 MB 00:00 2024-09-17T21:43:49.272 INFO:teuthology.orchestra.run.smithi165.stdout:(6/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 30 MB/s | 17 MB 00:00 2024-09-17T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T21:43:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T21:43:49.489 INFO:teuthology.orchestra.run.smithi165.stdout:(7/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 399 kB/s | 253 kB 00:00 2024-09-17T21:43:49.606 INFO:teuthology.orchestra.run.smithi165.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-17T21:43:49.723 INFO:teuthology.orchestra.run.smithi165.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.0 MB/s | 125 kB 00:00 2024-09-17T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T21:43:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:49.738 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:43:49.748 INFO:teuthology.orchestra.run.smithi165.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 224 kB/s | 106 kB 00:00 2024-09-17T21:43:49.832 INFO:teuthology.orchestra.run.smithi165.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-09-17T21:43:49.932 INFO:teuthology.orchestra.run.smithi165.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-17T21:43:49.983 INFO:teuthology.orchestra.run.smithi165.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 694 kB/s | 35 kB 00:00 2024-09-17T21:43:50.016 INFO:teuthology.orchestra.run.smithi165.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 935 kB/s | 172 kB 00:00 2024-09-17T21:43:50.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-09-17T21:43:50.042 INFO:teuthology.orchestra.run.smithi165.stdout:(15/37): python3-packaging-20.9-5.el9.noarch.rp 1.3 MB/s | 77 kB 00:00 2024-09-17T21:43:50.067 INFO:teuthology.orchestra.run.smithi165.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T21:43:50.092 INFO:teuthology.orchestra.run.smithi165.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-17T21:43:50.126 INFO:teuthology.orchestra.run.smithi165.stdout:(18/37): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:01 2024-09-17T21:43:50.176 INFO:teuthology.orchestra.run.smithi165.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-09-17T21:43:50.210 INFO:teuthology.orchestra.run.smithi165.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 572 kB/s | 43 kB 00:00 2024-09-17T21:43:50.235 INFO:teuthology.orchestra.run.smithi165.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.0 MB/s | 172 kB 00:00 2024-09-17T21:43:50.261 INFO:teuthology.orchestra.run.smithi165.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 426 kB/s | 11 kB 00:00 2024-09-17T21:43:50.286 INFO:teuthology.orchestra.run.smithi165.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 720 kB/s | 18 kB 00:00 2024-09-17T21:43:50.311 INFO:teuthology.orchestra.run.smithi165.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 937 kB/s | 23 kB 00:00 2024-09-17T21:43:50.337 INFO:teuthology.orchestra.run.smithi165.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-17T21:43:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 4/4 2024-09-17T21:43:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 1/4 2024-09-17T21:43:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2/4 2024-09-17T21:43:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-17T21:43:50.370 INFO:teuthology.orchestra.run.smithi165.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 331 kB/s | 19 kB 00:00 2024-09-17T21:43:50.395 INFO:teuthology.orchestra.run.smithi165.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 385 kB/s | 20 kB 00:00 2024-09-17T21:43:50.421 INFO:teuthology.orchestra.run.smithi165.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-17T21:43:50.446 INFO:teuthology.orchestra.run.smithi165.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-17T21:43:50.471 INFO:teuthology.orchestra.run.smithi165.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 662 kB/s | 16 kB 00:00 2024-09-17T21:43:50.497 INFO:teuthology.orchestra.run.smithi165.stdout:(31/37): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-09-17T21:43:50.530 INFO:teuthology.orchestra.run.smithi165.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-17T21:43:50.556 INFO:teuthology.orchestra.run.smithi165.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 611 kB/s | 36 kB 00:00 2024-09-17T21:43:50.581 INFO:teuthology.orchestra.run.smithi165.stdout:(34/37): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-17T21:43:50.615 INFO:teuthology.orchestra.run.smithi165.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 595 kB/s | 20 kB 00:00 2024-09-17T21:43:50.624 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:43:50.648 INFO:teuthology.orchestra.run.smithi165.stdout:(36/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-17T21:43:50.674 INFO:teuthology.orchestra.run.smithi165.stdout:(37/37): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-09-17T21:43:50.678 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:50.679 INFO:teuthology.orchestra.run.smithi165.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-17T21:43:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:50.681 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:43:50.764 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:43:50.766 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T21:43:50.767 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T21:43:50.768 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T21:43:50.769 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout:Install 37 Packages 2024-09-17T21:43:50.770 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 30 M 2024-09-17T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 108 M 2024-09-17T21:43:50.771 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:43:50.832 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:43:50.873 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:43:50.873 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:43:50.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph 2024-09-17T21:43:51.342 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:43:51.343 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:43:51.418 INFO:teuthology.orchestra.run.smithi007.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 54 kB/s | 6.5 kB 00:00 2024-09-17T21:43:51.560 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:41 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:43:51.627 INFO:teuthology.orchestra.run.smithi007.stdout:(2/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 7.2 MB/s | 2.4 MB 00:00 2024-09-17T21:43:51.669 INFO:teuthology.orchestra.run.smithi007.stdout:(3/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.1 MB/s | 1.5 MB 00:00 2024-09-17T21:43:51.701 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 6.5 k 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 2.4 M 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 1.5 M 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 249 k 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 4.7 M 2024-09-17T21:43:51.704 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 17 M 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T21:43:51.705 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T21:43:51.706 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout:Install 37 Packages 2024-09-17T21:43:51.707 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:51.708 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 30 M 2024-09-17T21:43:51.708 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 108 M 2024-09-17T21:43:51.708 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:43:51.765 INFO:teuthology.orchestra.run.smithi007.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.7 MB/s | 249 kB 00:00 2024-09-17T21:43:51.928 INFO:teuthology.orchestra.run.smithi007.stdout:(5/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 9.2 MB/s | 4.7 MB 00:00 2024-09-17T21:43:51.953 INFO:teuthology.orchestra.run.smithi007.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-17T21:43:51.996 INFO:teuthology.orchestra.run.smithi007.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-17T21:43:52.038 INFO:teuthology.orchestra.run.smithi007.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-17T21:43:52.213 INFO:teuthology.orchestra.run.smithi007.stdout:(9/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-17T21:43:52.238 INFO:teuthology.orchestra.run.smithi007.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 624 kB/s | 125 kB 00:00 2024-09-17T21:43:52.322 INFO:teuthology.orchestra.run.smithi007.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 3.1 MB/s | 1.2 MB 00:00 2024-09-17T21:43:52.364 INFO:teuthology.orchestra.run.smithi007.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.4 MB/s | 215 kB 00:00 2024-09-17T21:43:52.617 INFO:teuthology.orchestra.run.smithi053.stdout:(1/37): ceph-19.1.1-330.gc2ddf972.el9.x86_64.rp 46 kB/s | 6.5 kB 00:00 2024-09-17T21:43:52.740 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:43:52.784 INFO:teuthology.orchestra.run.smithi053.stdout:(2/37): ceph-mgr-19.1.1-330.gc2ddf972.el9.x86_6 4.9 MB/s | 1.5 MB 00:00 2024-09-17T21:43:52.802 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T21:43:52.834 INFO:teuthology.orchestra.run.smithi053.stdout:(3/37): ceph-mds-19.1.1-330.gc2ddf972.el9.x86_6 6.6 MB/s | 2.4 MB 00:00 2024-09-17T21:43:52.837 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T21:43:52.879 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T21:43:52.914 INFO:teuthology.orchestra.run.smithi007.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 63 kB/s | 35 kB 00:00 2024-09-17T21:43:52.918 INFO:teuthology.orchestra.run.smithi053.stdout:(4/37): ceph-mgr-modules-core-19.1.1-330.gc2ddf 2.9 MB/s | 249 kB 00:00 2024-09-17T21:43:52.950 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T21:43:52.972 INFO:teuthology.orchestra.run.smithi007.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 257 kB/s | 188 kB 00:00 2024-09-17T21:43:52.987 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T21:43:53.045 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T21:43:53.056 INFO:teuthology.orchestra.run.smithi007.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-09-17T21:43:53.102 INFO:teuthology.orchestra.run.smithi053.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-17T21:43:53.137 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T21:43:53.202 INFO:teuthology.orchestra.run.smithi053.stdout:(6/37): ceph-mon-19.1.1-330.gc2ddf972.el9.x86_6 8.1 MB/s | 4.7 MB 00:00 2024-09-17T21:43:53.215 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T21:43:53.264 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T21:43:53.299 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T21:43:53.335 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T21:43:53.369 INFO:teuthology.orchestra.run.smithi053.stdout:(7/37): ceph-osd-19.1.1-330.gc2ddf972.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-17T21:43:53.370 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T21:43:53.404 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T21:43:53.444 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T21:43:53.486 INFO:teuthology.orchestra.run.smithi053.stdout:(8/37): python3-cryptography-36.0.1-4.el9.x86_6 3.2 MB/s | 1.2 MB 00:00 2024-09-17T21:43:53.487 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T21:43:53.503 INFO:teuthology.orchestra.run.smithi053.stdout:(9/37): python3-ply-3.11-14.el9.noarch.rpm 354 kB/s | 106 kB 00:00 2024-09-17T21:43:53.522 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T21:43:53.528 INFO:teuthology.orchestra.run.smithi053.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 2.9 MB/s | 125 kB 00:00 2024-09-17T21:43:53.539 INFO:teuthology.orchestra.run.smithi007.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 123 kB/s | 77 kB 00:00 2024-09-17T21:43:53.562 INFO:teuthology.orchestra.run.smithi053.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 704 kB/s | 135 kB 00:00 2024-09-17T21:43:53.573 INFO:teuthology.orchestra.run.smithi007.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 85 kB/s | 51 kB 00:00 2024-09-17T21:43:53.578 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T21:43:53.578 INFO:teuthology.orchestra.run.smithi053.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-17T21:43:53.619 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T21:43:53.623 INFO:teuthology.orchestra.run.smithi007.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-17T21:43:53.640 INFO:teuthology.orchestra.run.smithi007.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-17T21:43:53.662 INFO:teuthology.orchestra.run.smithi053.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 416 kB/s | 35 kB 00:00 2024-09-17T21:43:53.665 INFO:teuthology.orchestra.run.smithi007.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.0 MB/s | 43 kB 00:00 2024-09-17T21:43:53.687 INFO:teuthology.orchestra.run.smithi053.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-09-17T21:43:53.693 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T21:43:53.699 INFO:teuthology.orchestra.run.smithi007.stdout:(21/37): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-09-17T21:43:53.713 INFO:teuthology.orchestra.run.smithi053.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T21:43:53.738 INFO:teuthology.orchestra.run.smithi053.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T21:43:53.758 INFO:teuthology.orchestra.run.smithi007.stdout:(22/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2024-09-17T21:43:53.763 INFO:teuthology.orchestra.run.smithi053.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 767 kB/s | 77 kB 00:00 2024-09-17T21:43:53.783 INFO:teuthology.orchestra.run.smithi007.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 131 kB/s | 11 kB 00:00 2024-09-17T21:43:53.788 INFO:teuthology.orchestra.run.smithi053.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-09-17T21:43:53.808 INFO:teuthology.orchestra.run.smithi007.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 367 kB/s | 18 kB 00:00 2024-09-17T21:43:53.834 INFO:teuthology.orchestra.run.smithi007.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-17T21:43:53.834 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T21:43:53.839 INFO:teuthology.orchestra.run.smithi053.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-17T21:43:53.859 INFO:teuthology.orchestra.run.smithi007.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-17T21:43:53.872 INFO:teuthology.orchestra.run.smithi053.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 521 kB/s | 43 kB 00:00 2024-09-17T21:43:53.884 INFO:teuthology.orchestra.run.smithi007.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-17T21:43:53.898 INFO:teuthology.orchestra.run.smithi053.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-17T21:43:53.909 INFO:teuthology.orchestra.run.smithi007.stdout:(28/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 639 kB/s | 172 kB 00:00 2024-09-17T21:43:53.911 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T21:43:53.923 INFO:teuthology.orchestra.run.smithi053.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-17T21:43:53.935 INFO:teuthology.orchestra.run.smithi007.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 615 kB/s | 46 kB 00:00 2024-09-17T21:43:53.948 INFO:teuthology.orchestra.run.smithi053.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.8 MB/s | 358 kB 00:00 2024-09-17T21:43:53.960 INFO:teuthology.orchestra.run.smithi007.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-17T21:43:53.964 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T21:43:53.973 INFO:teuthology.orchestra.run.smithi053.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 351 kB/s | 18 kB 00:00 2024-09-17T21:43:53.985 INFO:teuthology.orchestra.run.smithi007.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-17T21:43:53.999 INFO:teuthology.orchestra.run.smithi053.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-17T21:43:54.003 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T21:43:54.011 INFO:teuthology.orchestra.run.smithi007.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-17T21:43:54.024 INFO:teuthology.orchestra.run.smithi053.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 308 kB/s | 23 kB 00:00 2024-09-17T21:43:54.036 INFO:teuthology.orchestra.run.smithi007.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-17T21:43:54.049 INFO:teuthology.orchestra.run.smithi053.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-17T21:43:54.059 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T21:43:54.061 INFO:teuthology.orchestra.run.smithi007.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-17T21:43:54.074 INFO:teuthology.orchestra.run.smithi053.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-17T21:43:54.095 INFO:teuthology.orchestra.run.smithi007.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-17T21:43:54.099 INFO:teuthology.orchestra.run.smithi053.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-17T21:43:54.115 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T21:43:54.116 INFO:teuthology.orchestra.run.smithi053.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2024-09-17T21:43:54.120 INFO:teuthology.orchestra.run.smithi007.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-17T21:43:54.137 INFO:teuthology.orchestra.run.smithi053.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 445 kB/s | 16 kB 00:00 2024-09-17T21:43:54.145 INFO:teuthology.orchestra.run.smithi007.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-17T21:43:54.151 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:54.151 INFO:teuthology.orchestra.run.smithi007.stdout:Total 9.0 MB/s | 30 MB 00:03 2024-09-17T21:43:54.162 INFO:teuthology.orchestra.run.smithi053.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-17T21:43:54.187 INFO:teuthology.orchestra.run.smithi053.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-17T21:43:54.213 INFO:teuthology.orchestra.run.smithi053.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-17T21:43:54.238 INFO:teuthology.orchestra.run.smithi053.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-17T21:43:54.263 INFO:teuthology.orchestra.run.smithi053.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T21:43:54.288 INFO:teuthology.orchestra.run.smithi053.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-17T21:43:54.292 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:43:54.292 INFO:teuthology.orchestra.run.smithi053.stdout:Total 12 MB/s | 30 MB 00:02 2024-09-17T21:43:54.309 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:43:54.314 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T21:43:54.354 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:43:54.354 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:43:54.366 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T21:43:54.444 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T21:43:54.454 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:43:54.499 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:43:54.499 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:43:54.533 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T21:43:54.585 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T21:43:54.709 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T21:43:54.858 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:43:54.859 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:43:54.878 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T21:43:54.909 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:54.935 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:54.935 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:54.936 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T21:43:54.936 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:54.936 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:54.936 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:55.002 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:43:55.003 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:43:56.253 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:43:56.319 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T21:43:56.352 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T21:43:56.376 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:43:56.379 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T21:43:56.404 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:43:56.404 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:56.404 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T21:43:56.404 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:43:56.404 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:43:56.405 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:56.418 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:43:56.456 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T21:43:56.500 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-17T21:43:56.504 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T21:43:56.537 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T21:43:56.549 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-17T21:43:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-17T21:43:56.635 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T21:43:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-17T21:43:56.693 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-17T21:43:56.719 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T21:43:56.726 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-17T21:43:56.766 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T21:43:56.800 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T21:43:56.822 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-17T21:43:56.838 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T21:43:56.844 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:43:56.868 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:43:56.868 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:56.869 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T21:43:56.869 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:43:56.869 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:43:56.869 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:56.872 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T21:43:56.899 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-17T21:43:56.905 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T21:43:56.947 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T21:43:56.954 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-17T21:43:56.989 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-17T21:43:56.990 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T21:43:57.018 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-17T21:43:57.024 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T21:43:57.053 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-17T21:43:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T21:43:57.086 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-17T21:43:57.095 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:43:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T21:43:57.119 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:43:57.119 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:57.120 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T21:43:57.120 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:43:57.120 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:43:57.120 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:43:57.127 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-17T21:43:57.170 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-17T21:43:57.184 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:43:57.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T21:43:57.204 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-17T21:43:57.253 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-17T21:43:57.297 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-17T21:43:57.353 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T21:43:57.371 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-17T21:43:57.442 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T21:43:57.500 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T21:43:57.530 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-17T21:43:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T21:43:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T21:43:57.613 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-17T21:43:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T21:43:57.672 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-17T21:43:57.717 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-17T21:43:57.776 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-17T21:43:57.840 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-17T21:43:57.866 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T21:43:57.923 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T21:43:58.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T21:43:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-17T21:43:58.088 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-17T21:43:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T21:43:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T21:43:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-17T21:43:58.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-17T21:43:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T21:43:58.297 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-17T21:43:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-17T21:43:58.459 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T21:43:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T21:43:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:43:58.598 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 32/37 2024-09-17T21:43:58.629 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 33/37 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T21:43:58.658 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T21:43:59.098 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T21:43:59.099 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T21:43:59.100 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T21:43:59.100 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T21:43:59.100 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T21:43:59.100 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T21:43:59.100 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T21:43:59.101 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T21:44:00.041 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:44:00.070 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T21:44:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:00.097 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:00.097 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T21:44:00.098 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T21:44:00.099 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:00.100 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:44:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 34/37 2024-09-17T21:44:00.210 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.210 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T21:44:00.210 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:44:00.210 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T21:44:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:00.533 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:44:00.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:00.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-base 2024-09-17T21:44:00.664 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:44:00.689 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 35/37 2024-09-17T21:44:00.690 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.690 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T21:44:00.690 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:44:00.690 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T21:44:00.690 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:44:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:00.892 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:44:00.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:44:00.957 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 36/37 2024-09-17T21:44:00.957 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:00.957 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T21:44:00.957 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:44:00.957 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T21:44:00.958 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:01.023 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:44:01.159 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:01.248 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:01.289 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:01.291 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:01.291 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:01.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install cephadm 2024-09-17T21:44:01.942 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:52 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:02.074 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:02.075 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:02.076 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 765 k 2024-09-17T21:44:02.076 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 771 k 2024-09-17T21:44:02.076 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:02.337 INFO:teuthology.orchestra.run.smithi165.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-17T21:44:02.337 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:02.338 INFO:teuthology.orchestra.run.smithi165.stdout:Total 2.8 MB/s | 765 kB 00:00 2024-09-17T21:44:02.338 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:02.343 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:02.343 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:02.349 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:02.350 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:02.423 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T21:44:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T21:44:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T21:44:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T21:44:02.789 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 37/37 2024-09-17T21:44:02.852 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 1/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 3/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 4/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 5/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.n 6/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-17T21:44:02.853 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-17T21:44:02.854 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-17T21:44:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-17T21:44:02.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-17T21:44:02.856 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-17T21:44:02.882 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:03.272 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:03.546 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:03.547 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:03.547 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:03.547 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:03.547 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:03.547 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:03.713 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T21:44:03.713 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T21:44:03.714 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T21:44:03.715 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:03.716 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:03.774 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:03.861 INFO:teuthology.orchestra.run.smithi053.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T21:44:03.862 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T21:44:03.863 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T21:44:03.864 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T21:44:03.864 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T21:44:03.864 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T21:44:03.864 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:03.864 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:04.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-09-17T21:44:04.265 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-base 2024-09-17T21:44:04.367 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:04.502 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:04.502 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:04.502 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:04.503 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 145 k 2024-09-17T21:44:04.504 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 435 k 2024-09-17T21:44:04.504 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:04.676 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:04.711 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 699 kB/s | 145 kB 00:00 2024-09-17T21:44:04.712 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:04.713 INFO:teuthology.orchestra.run.smithi165.stdout:Total 692 kB/s | 145 kB 00:00 2024-09-17T21:44:04.713 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:04.720 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:04.720 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:04.768 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:04.777 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:04.777 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:04.813 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:04.814 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:04.814 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:04.874 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:54 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:04.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-09-17T21:44:04.939 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:04.966 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-base-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:05.012 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:05.013 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:05.014 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:05.054 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:05.080 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:05.080 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:05.080 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T21:44:05.080 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:05.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install cephadm 2024-09-17T21:44:05.491 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:05.627 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:05.628 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 765 k 2024-09-17T21:44:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 771 k 2024-09-17T21:44:05.629 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:05.694 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:55 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:05.831 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 765 k 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:05.832 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 765 k 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 771 k 2024-09-17T21:44:05.833 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:05.908 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.7 MB/s | 765 kB 00:00 2024-09-17T21:44:05.908 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:05.909 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.7 MB/s | 765 kB 00:00 2024-09-17T21:44:05.909 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:05.915 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:05.915 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:05.921 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:05.921 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:05.987 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:06.088 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm-19.1.1-330.gc2ddf972.el9.noarch.rpm 2.9 MB/s | 765 kB 00:00 2024-09-17T21:44:06.089 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:06.089 INFO:teuthology.orchestra.run.smithi053.stdout:Total 2.9 MB/s | 765 kB 00:00 2024-09-17T21:44:06.089 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:06.095 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:06.095 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:06.101 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:06.101 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:06.176 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:06.247 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr 2024-09-17T21:44:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:06.568 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:06.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:06.669 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:06.834 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:57 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:06.926 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:06.968 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:06.969 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:06.969 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:07.053 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T21:44:07.156 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:07.198 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:07.198 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:07.199 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:07.199 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:07.403 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T21:44:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:07.422 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:07.422 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:07.637 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:07.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-immutable-object-cache 2024-09-17T21:44:07.770 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:07.771 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:07.771 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T21:44:07.772 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout:Install 12 Packages 2024-09-17T21:44:07.773 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:07.774 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 8.7 M 2024-09-17T21:44:07.774 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 104 M 2024-09-17T21:44:07.774 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:08.011 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:59 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:08.100 INFO:teuthology.orchestra.run.smithi165.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 221 kB/s | 32 kB 00:00 2024-09-17T21:44:08.125 INFO:teuthology.orchestra.run.smithi165.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 97 kB/s | 16 kB 00:00 2024-09-17T21:44:08.153 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:08.154 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 145 k 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 435 k 2024-09-17T21:44:08.155 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:08.256 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:58 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:08.325 INFO:teuthology.orchestra.run.smithi165.stdout:(3/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 9.6 MB/s | 3.6 MB 00:00 2024-09-17T21:44:08.350 INFO:teuthology.orchestra.run.smithi165.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-17T21:44:08.390 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 620 kB/s | 145 kB 00:00 2024-09-17T21:44:08.390 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:08.390 INFO:teuthology.orchestra.run.smithi007.stdout:Total 616 kB/s | 145 kB 00:00 2024-09-17T21:44:08.391 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:08.393 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 145 k 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:08.394 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:08.395 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 145 k 2024-09-17T21:44:08.395 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 435 k 2024-09-17T21:44:08.395 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:08.399 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:08.399 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:08.425 INFO:teuthology.orchestra.run.smithi165.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 7.1 MB/s | 548 kB 00:00 2024-09-17T21:44:08.459 INFO:teuthology.orchestra.run.smithi165.stdout:(6/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-17T21:44:08.460 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:08.460 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:08.476 INFO:teuthology.orchestra.run.smithi165.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 386 kB/s | 19 kB 00:00 2024-09-17T21:44:08.510 INFO:teuthology.orchestra.run.smithi165.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.3 MB/s | 144 kB 00:00 2024-09-17T21:44:08.527 INFO:teuthology.orchestra.run.smithi165.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 2.0 MB/s | 31 kB 00:00 2024-09-17T21:44:08.561 INFO:teuthology.orchestra.run.smithi165.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-17T21:44:08.608 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:08.612 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-immutable-object-cache-19.1.1-330.gc2ddf97 670 kB/s | 145 kB 00:00 2024-09-17T21:44:08.613 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:08.613 INFO:teuthology.orchestra.run.smithi053.stdout:Total 665 kB/s | 145 kB 00:00 2024-09-17T21:44:08.613 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:08.621 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:08.622 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:08.653 INFO:teuthology.orchestra.run.smithi165.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2024-09-17T21:44:08.665 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:08.678 INFO:teuthology.orchestra.run.smithi165.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.4 MB/s | 863 kB 00:00 2024-09-17T21:44:08.681 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:08.681 INFO:teuthology.orchestra.run.smithi165.stdout:Total 9.5 MB/s | 8.7 MB 00:00 2024-09-17T21:44:08.682 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:08.682 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:08.692 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:08.692 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:08.692 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T21:44:08.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:08.730 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:08.747 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:08.747 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:08.843 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:08.897 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:08.923 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:08.923 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:08.923 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T21:44:08.923 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:08.960 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:08.960 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:09.506 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:09.579 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T21:44:09.622 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:09.644 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:09.669 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T21:44:09.708 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T21:44:09.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T21:44:09.855 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.e 1/1 2024-09-17T21:44:09.855 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:09.855 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:09.855 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:09.856 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:09.856 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:09.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-09-17T21:44:09.943 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T21:44:10.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T21:44:10.080 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T21:44:10.128 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:10.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr 2024-09-17T21:44:10.139 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T21:44:10.528 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:10.621 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:10.665 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:10.666 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:10.666 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:10.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T21:44:10.737 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:00 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-mgr-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:10.873 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:10.874 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:10.874 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:10.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-dashboard 2024-09-17T21:44:11.200 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T21:44:11.240 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:11.340 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:02 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:11.477 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T21:44:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:11.480 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:11.481 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:11.481 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-17T21:44:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:11.481 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 8.7 M 2024-09-17T21:44:11.481 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 104 M 2024-09-17T21:44:11.482 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:11.551 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:01 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:11.688 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:11.689 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:11.689 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 3.6 M 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 32 k 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 16 k 2024-09-17T21:44:11.690 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:11.691 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:11.692 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-17T21:44:11.692 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:11.692 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 8.7 M 2024-09-17T21:44:11.692 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 104 M 2024-09-17T21:44:11.692 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:11.994 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 227 kB/s | 32 kB 00:00 2024-09-17T21:44:12.019 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 99 kB/s | 16 kB 00:00 2024-09-17T21:44:12.227 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 9.5 MB/s | 3.6 MB 00:00 2024-09-17T21:44:12.249 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): ceph-grafana-dashboards-19.1.1-330.gc2d 206 kB/s | 32 kB 00:00 2024-09-17T21:44:12.274 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): ceph-prometheus-alerts-19.1.1-330.gc2dd 91 kB/s | 16 kB 00:00 2024-09-17T21:44:12.345 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 7.2 MB/s | 863 kB 00:00 2024-09-17T21:44:12.428 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 6.4 MB/s | 548 kB 00:00 2024-09-17T21:44:12.450 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-17T21:44:12.453 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-17T21:44:12.508 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): ceph-mgr-dashboard-19.1.1-330.gc2ddf972 8.6 MB/s | 3.6 MB 00:00 2024-09-17T21:44:12.542 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-09-17T21:44:12.570 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 18 MB/s | 2.0 MB 00:00 2024-09-17T21:44:12.604 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-17T21:44:12.617 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.9 MB/s | 548 kB 00:00 2024-09-17T21:44:12.621 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.8 MB/s | 31 kB 00:00 2024-09-17T21:44:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T21:44:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T21:44:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T21:44:12.640 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T21:44:12.641 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T21:44:12.662 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-17T21:44:12.684 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.6 MB/s | 863 kB 00:00 2024-09-17T21:44:12.688 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-protobuf-3.14.0-13.el9.noarch. 403 kB/s | 269 kB 00:00 2024-09-17T21:44:12.717 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): grpc-data-1.46.7-10.el9.noarch.rpm 116 kB/s | 19 kB 00:00 2024-09-17T21:44:12.760 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.9 MB/s | 144 kB 00:00 2024-09-17T21:44:12.785 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 473 kB/s | 31 kB 00:00 2024-09-17T21:44:12.827 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-17T21:44:12.868 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.1 MB/s | 2.0 MB 00:00 2024-09-17T21:44:12.873 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:12.873 INFO:teuthology.orchestra.run.smithi053.stdout:Total 7.3 MB/s | 8.7 MB 00:01 2024-09-17T21:44:12.926 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:12.946 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:12.946 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T21:44:12.961 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:12.962 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:13.048 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 982 kB/s | 1.0 MB 00:01 2024-09-17T21:44:13.050 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:13.050 INFO:teuthology.orchestra.run.smithi007.stdout:Total 5.5 MB/s | 8.7 MB 00:01 2024-09-17T21:44:13.104 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:13.123 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:13.123 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:13.155 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:13.155 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:13.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T21:44:13.343 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:13.343 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:13.795 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T21:44:13.809 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:04 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T21:44:13.885 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T21:44:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:13.923 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T21:44:13.942 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:13.943 INFO:teuthology.orchestra.run.smithi165.stdout:====================================================================================== 2024-09-17T21:44:13.943 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:13.943 INFO:teuthology.orchestra.run.smithi165.stdout:====================================================================================== 2024-09-17T21:44:13.943 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T21:44:13.944 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout:====================================================================================== 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout:Install 12 Packages 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 38 M 2024-09-17T21:44:13.945 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 212 M 2024-09-17T21:44:13.946 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-17T21:44:14.033 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-17T21:44:14.035 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T21:44:14.081 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-17T21:44:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-17T21:44:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T21:44:14.213 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-17T21:44:14.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T21:44:14.303 INFO:teuthology.orchestra.run.smithi165.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-17T21:44:14.320 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T21:44:14.345 INFO:teuthology.orchestra.run.smithi165.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-17T21:44:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-17T21:44:14.356 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:14.388 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T21:44:14.450 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-17T21:44:14.508 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-17T21:44:14.535 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 10/12 2024-09-17T21:44:14.629 INFO:teuthology.orchestra.run.smithi165.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T21:44:14.645 INFO:teuthology.orchestra.run.smithi165.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 95 kB/s | 33 kB 00:00 2024-09-17T21:44:14.837 INFO:teuthology.orchestra.run.smithi165.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 201 kB/s | 38 kB 00:00 2024-09-17T21:44:14.913 INFO:teuthology.orchestra.run.smithi165.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-17T21:44:15.371 INFO:teuthology.orchestra.run.smithi165.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 534 kB/s | 245 kB 00:00 2024-09-17T21:44:15.430 INFO:teuthology.orchestra.run.smithi165.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-17T21:44:15.501 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T21:44:15.538 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:15.547 INFO:teuthology.orchestra.run.smithi165.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-17T21:44:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 11/12 2024-09-17T21:44:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:16.181 INFO:teuthology.orchestra.run.smithi165.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.3 MB/s | 5.1 MB 00:00 2024-09-17T21:44:16.314 INFO:teuthology.orchestra.run.smithi165.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 3.3 MB/s | 4.9 MB 00:01 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T21:44:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T21:44:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T21:44:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T21:44:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 12/12 2024-09-17T21:44:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9 1/12 2024-09-17T21:44:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noar 2/12 2024-09-17T21:44:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9. 3/12 2024-09-17T21:44:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-17T21:44:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-17T21:44:17.147 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T21:44:17.147 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:17.147 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:17.147 INFO:teuthology.orchestra.run.smithi053.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T21:44:17.147 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:17.148 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:17.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T21:44:17.393 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-17T21:44:17.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:17.393 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T21:44:17.394 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:17.395 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:17.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-17T21:44:17.672 INFO:teuthology.orchestra.run.smithi165.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.6 MB/s | 16 MB 00:02 2024-09-17T21:44:17.673 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:17.674 INFO:teuthology.orchestra.run.smithi165.stdout:Total 10 MB/s | 38 MB 00:03 2024-09-17T21:44:18.000 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:07 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:18.029 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:18.056 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:18.056 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:18.141 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T21:44:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-17T21:44:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 38 M 2024-09-17T21:44:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 212 M 2024-09-17T21:44:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:18.246 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:09 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 7.4 M 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T21:44:18.392 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:====================================================================================== 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-09-17T21:44:18.393 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 38 M 2024-09-17T21:44:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 212 M 2024-09-17T21:44:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:18.491 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:18.492 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:18.615 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-17T21:44:18.673 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-17T21:44:18.692 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): ceph-mgr-diskprediction-local-19.1.1-33 16 MB/s | 7.4 MB 00:00 2024-09-17T21:44:18.784 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 349 kB/s | 191 kB 00:00 2024-09-17T21:44:18.809 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-17T21:44:18.859 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-17T21:44:18.918 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 659 kB/s | 38 kB 00:00 2024-09-17T21:44:18.931 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): ceph-mgr-diskprediction-local-19.1.1-33 15 MB/s | 7.4 MB 00:00 2024-09-17T21:44:18.957 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 96 kB/s | 33 kB 00:00 2024-09-17T21:44:18.976 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-17T21:44:19.069 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-17T21:44:19.165 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 185 kB/s | 38 kB 00:00 2024-09-17T21:44:19.185 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-09-17T21:44:19.210 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2024-09-17T21:44:19.224 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 62 kB/s | 18 kB 00:00 2024-09-17T21:44:19.236 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.9 MB/s | 250 kB 00:00 2024-09-17T21:44:19.281 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:19.311 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 13 MB/s | 4.9 MB 00:00 2024-09-17T21:44:19.357 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T21:44:19.395 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:19.495 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:19.538 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T21:44:19.599 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 652 kB/s | 245 kB 00:00 2024-09-17T21:44:19.637 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-17T21:44:19.637 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:19.637 INFO:teuthology.orchestra.run.smithi053.stdout:Total 26 MB/s | 38 MB 00:01 2024-09-17T21:44:19.724 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T21:44:19.754 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T21:44:19.766 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-17T21:44:19.841 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T21:44:19.891 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-17T21:44:20.005 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:20.033 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:20.033 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:20.275 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.5 MB/s | 4.9 MB 00:01 2024-09-17T21:44:20.491 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:20.491 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:20.492 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T21:44:20.525 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-09-17T21:44:20.609 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T21:44:21.397 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:21.467 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T21:44:21.510 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:21.614 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:21.653 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T21:44:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T21:44:21.852 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T21:44:21.955 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T21:44:22.333 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 6.7 MB/s | 16 MB 00:02 2024-09-17T21:44:22.335 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:22.335 INFO:teuthology.orchestra.run.smithi007.stdout:Total 9.7 MB/s | 38 MB 00:03 2024-09-17T21:44:22.648 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T21:44:22.706 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:22.735 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:22.735 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:22.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T21:44:22.889 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T21:44:22.913 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:23.189 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:23.190 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:24.003 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:24.084 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-17T21:44:24.106 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:24.124 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:24.231 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:24.267 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-17T21:44:24.458 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-17T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-17T21:44:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-17T21:44:25.181 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T21:44:25.208 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:25.292 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-17T21:44:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-17T21:44:25.532 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:25.532 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T21:44:25.532 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:25.533 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T21:44:25.945 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T21:44:25.945 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:25.945 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:25.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:25.946 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:26.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-rook 2024-09-17T21:44:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:26.984 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:17 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:27.117 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T21:44:27.118 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T21:44:27.119 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout:Install 13 Packages 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 2.2 M 2024-09-17T21:44:27.120 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 26 M 2024-09-17T21:44:27.121 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:27.257 INFO:teuthology.orchestra.run.smithi165.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 308 kB/s | 26 kB 00:00 2024-09-17T21:44:27.282 INFO:teuthology.orchestra.run.smithi165.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 176 kB/s | 19 kB 00:00 2024-09-17T21:44:27.325 INFO:teuthology.orchestra.run.smithi165.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.3 MB/s | 222 kB 00:00 2024-09-17T21:44:27.375 INFO:teuthology.orchestra.run.smithi165.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-17T21:44:27.400 INFO:teuthology.orchestra.run.smithi165.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.6 MB/s | 279 kB 00:00 2024-09-17T21:44:27.434 INFO:teuthology.orchestra.run.smithi165.stdout:(6/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 189 kB/s | 49 kB 00:00 2024-09-17T21:44:27.460 INFO:teuthology.orchestra.run.smithi165.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 636 kB/s | 54 kB 00:00 2024-09-17T21:44:27.510 INFO:teuthology.orchestra.run.smithi165.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-09-17T21:44:27.535 INFO:teuthology.orchestra.run.smithi165.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-17T21:44:27.560 INFO:teuthology.orchestra.run.smithi165.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T21:44:27.586 INFO:teuthology.orchestra.run.smithi165.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-17T21:44:27.611 INFO:teuthology.orchestra.run.smithi165.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T21:44:27.644 INFO:teuthology.orchestra.run.smithi165.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-17T21:44:27.648 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:27.648 INFO:teuthology.orchestra.run.smithi165.stdout:Total 4.2 MB/s | 2.2 MB 00:00 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T21:44:27.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T21:44:27.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T21:44:27.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:27.677 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T21:44:27.711 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:27.726 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:27.726 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:27.818 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:27.820 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:27.837 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-17T21:44:27.882 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:28.113 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T21:44:28.113 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:28.113 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:28.114 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:28.331 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:28.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-rook 2024-09-17T21:44:28.491 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T21:44:28.579 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T21:44:28.635 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T21:44:28.672 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T21:44:28.713 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T21:44:28.814 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T21:44:28.915 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T21:44:28.966 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T21:44:29.065 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:19 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:29.203 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T21:44:29.204 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T21:44:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 2.2 M 2024-09-17T21:44:29.206 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 26 M 2024-09-17T21:44:29.207 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:29.674 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T21:44:29.721 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T21:44:29.754 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T21:44:29.766 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 279 kB/s | 49 kB 00:00 2024-09-17T21:44:29.803 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T21:44:29.832 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:29.875 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 67 kB/s | 19 kB 00:00 2024-09-17T21:44:29.900 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 83 kB/s | 26 kB 00:00 2024-09-17T21:44:30.242 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 433 kB/s | 159 kB 00:00 2024-09-17T21:44:30.334 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 391 kB/s | 222 kB 00:00 2024-09-17T21:44:30.368 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 428 kB/s | 54 kB 00:00 2024-09-17T21:44:30.393 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 566 kB/s | 279 kB 00:00 2024-09-17T21:44:30.443 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 297 kB/s | 32 kB 00:00 2024-09-17T21:44:30.468 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 141 kB/s | 14 kB 00:00 2024-09-17T21:44:30.502 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-17T21:44:30.527 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-17T21:44:30.552 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-17T21:44:30.586 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-17T21:44:30.590 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:30.590 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T21:44:30.604 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T21:44:30.605 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T21:44:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 12/12 2024-09-17T21:44:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf9 1/12 2024-09-17T21:44:30.610 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-17T21:44:30.611 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-17T21:44:30.662 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:30.680 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:30.680 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:30.781 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:30.782 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:30.932 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T21:44:30.932 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T21:44:30.933 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T21:44:30.934 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T21:44:30.934 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T21:44:30.934 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:30.934 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T21:44:30.999 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:31.000 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:31.235 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T21:44:31.304 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:31.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-09-17T21:44:31.485 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T21:44:31.539 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T21:44:31.595 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T21:44:31.630 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T21:44:31.673 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T21:44:31.779 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T21:44:31.817 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:31.948 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T21:44:31.949 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:Install 6 Packages 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 7.0 M 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 33 M 2024-09-17T21:44:31.950 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:31.998 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T21:44:32.000 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:22 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:32.049 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T21:44:32.139 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:32.140 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:32.140 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:32.140 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 49 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T21:44:32.141 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-09-17T21:44:32.142 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:32.143 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.2 M 2024-09-17T21:44:32.143 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-09-17T21:44:32.143 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:32.239 INFO:teuthology.orchestra.run.smithi165.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 699 kB/s | 146 kB 00:00 2024-09-17T21:44:32.306 INFO:teuthology.orchestra.run.smithi165.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.1 MB/s | 548 kB 00:00 2024-09-17T21:44:32.331 INFO:teuthology.orchestra.run.smithi165.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-17T21:44:32.356 INFO:teuthology.orchestra.run.smithi165.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.0 MB/s | 74 kB 00:00 2024-09-17T21:44:32.716 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 240 kB/s | 19 kB 00:00 2024-09-17T21:44:32.742 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 244 kB/s | 26 kB 00:00 2024-09-17T21:44:32.792 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-09-17T21:44:32.806 INFO:teuthology.orchestra.run.smithi165.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 320 kB/s | 248 kB 00:00 2024-09-17T21:44:32.825 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): ceph-mgr-rook-19.1.1-330.gc2ddf972.el9. 261 kB/s | 49 kB 00:00 2024-09-17T21:44:32.830 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T21:44:32.851 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-17T21:44:32.876 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-09-17T21:44:32.879 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T21:44:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T21:44:32.918 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-09-17T21:44:32.943 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-17T21:44:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T21:44:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:32.993 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 128 kB/s | 14 kB 00:00 2024-09-17T21:44:33.027 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-17T21:44:33.061 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-17T21:44:33.086 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.3 MB/s | 220 kB 00:00 2024-09-17T21:44:33.128 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-17T21:44:33.132 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:33.132 INFO:teuthology.orchestra.run.smithi007.stdout:Total 2.2 MB/s | 2.2 MB 00:00 2024-09-17T21:44:33.200 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:33.216 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:33.216 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:33.319 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:33.320 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:33.474 INFO:teuthology.orchestra.run.smithi165.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-09-17T21:44:33.476 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:33.476 INFO:teuthology.orchestra.run.smithi165.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-17T21:44:33.549 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:33.561 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:33.561 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:33.660 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:33.660 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T21:44:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T21:44:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T21:44:33.955 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:33.971 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:34.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-17T21:44:34.142 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T21:44:34.189 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-17T21:44:34.192 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T21:44:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T21:44:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:34.246 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-17T21:44:34.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-17T21:44:34.322 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-17T21:44:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-17T21:44:34.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T21:44:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-17T21:44:34.591 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-17T21:44:34.718 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T21:44:34.796 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T21:44:34.860 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T21:44:34.888 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:35.113 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:25 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:35.252 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T21:44:35.253 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-09-17T21:44:35.254 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:35.255 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 7.0 M 2024-09-17T21:44:35.255 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 33 M 2024-09-17T21:44:35.255 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-17T21:44:35.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-17T21:44:35.430 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-17T21:44:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-17T21:44:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T21:44:35.639 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T21:44:35.654 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 703 kB/s | 146 kB 00:00 2024-09-17T21:44:35.738 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-17T21:44:35.771 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.7 MB/s | 58 kB 00:00 2024-09-17T21:44:35.797 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T21:44:35.931 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:35.932 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:35.988 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 457 kB/s | 248 kB 00:00 2024-09-17T21:44:36.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-fuse 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 13/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 1/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-17T21:44:36.352 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-17T21:44:36.353 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-17T21:44:36.768 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:27 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T21:44:36.824 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:36.825 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:36.901 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:36.901 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:36.901 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:36.901 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:36.901 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2024-09-17T21:44:36.902 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:36.903 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 977 k 2024-09-17T21:44:36.903 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 2.9 M 2024-09-17T21:44:36.903 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:37.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-09-17T21:44:37.276 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 628 kB/s | 80 kB 00:00 2024-09-17T21:44:37.451 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 2.9 MB/s | 897 kB 00:00 2024-09-17T21:44:37.451 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:37.452 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.7 MB/s | 977 kB 00:00 2024-09-17T21:44:37.463 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:37.485 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:37.485 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:37.566 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:37.566 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:37.703 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:28 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:37.755 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:37.841 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:37.841 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:37.841 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 146 k 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:37.842 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:37.843 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:37.843 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-09-17T21:44:37.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:37.843 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 7.0 M 2024-09-17T21:44:37.844 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-09-17T21:44:37.844 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:37.906 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T21:44:37.983 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:38.130 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-17T21:44:38.156 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): ceph-mgr-cephadm-19.1.1-330.gc2ddf972.el 640 kB/s | 146 kB 00:00 2024-09-17T21:44:38.215 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 987 kB/s | 58 kB 00:00 2024-09-17T21:44:38.239 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-17T21:44:38.282 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.5 MB/s | 548 kB 00:00 2024-09-17T21:44:38.660 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:38.661 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T21:44:38.872 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.7 MB/s | 6.0 MB 00:03 2024-09-17T21:44:38.874 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:38.874 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.9 MB/s | 7.0 MB 00:03 2024-09-17T21:44:38.939 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T21:44:38.939 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:38.939 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:38.939 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T21:44:38.939 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:38.940 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:38.950 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:38.963 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:38.963 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:39.070 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:39.070 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:39.090 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-09-17T21:44:39.092 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:39.093 INFO:teuthology.orchestra.run.smithi007.stdout:Total 5.6 MB/s | 7.0 MB 00:01 2024-09-17T21:44:39.166 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:39.178 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:39.178 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:39.239 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-volume 2024-09-17T21:44:39.284 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:39.284 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:39.561 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T21:44:39.597 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T21:44:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-17T21:44:39.818 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-17T21:44:39.842 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:30 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:39.977 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:39.977 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:39.977 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:39.978 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 285 k 2024-09-17T21:44:39.979 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 1.3 M 2024-09-17T21:44:39.979 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T21:44:40.224 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.1 MB/s | 285 kB 00:00 2024-09-17T21:44:40.224 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:40.225 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-17T21:44:40.225 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:40.233 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:40.234 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:40.254 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T21:44:40.258 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:40.258 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:40.326 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T21:44:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:40.370 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-17T21:44:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-17T21:44:40.475 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:40.536 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-17T21:44:40.551 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:40.562 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:40.566 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T21:44:40.567 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T21:44:41.125 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T21:44:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 6/6 2024-09-17T21:44:41.369 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 1/6 2024-09-17T21:44:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-17T21:44:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-17T21:44:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-17T21:44:41.370 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T21:44:41.418 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T21:44:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T21:44:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T21:44:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T21:44:41.419 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:41.419 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:41.572 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:41.572 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:41.573 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T21:44:41.648 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T21:44:41.649 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T21:44:41.649 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:41.649 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:41.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-fuse 2024-09-17T21:44:41.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-09-17T21:44:41.906 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados-devel 2024-09-17T21:44:42.275 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:42.414 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:42.414 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:42.414 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-09-17T21:44:42.415 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:42.416 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 977 k 2024-09-17T21:44:42.416 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 2.9 M 2024-09-17T21:44:42.416 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:42.499 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:42.505 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:42.556 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 864 kB/s | 80 kB 00:00 2024-09-17T21:44:42.636 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 897 k 2024-09-17T21:44:42.637 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-09-17T21:44:42.638 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 977 k 2024-09-17T21:44:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.9 M 2024-09-17T21:44:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:42.640 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:42.640 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:42.640 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-09-17T21:44:42.640 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:42.641 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:42.642 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 127 k 2024-09-17T21:44:42.642 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 456 k 2024-09-17T21:44:42.642 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:42.731 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.3 MB/s | 897 kB 00:00 2024-09-17T21:44:42.732 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:42.732 INFO:teuthology.orchestra.run.smithi053.stdout:Total 3.0 MB/s | 977 kB 00:00 2024-09-17T21:44:42.744 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:42.768 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:42.830 INFO:teuthology.orchestra.run.smithi165.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 672 kB/s | 127 kB 00:00 2024-09-17T21:44:42.830 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:42.831 INFO:teuthology.orchestra.run.smithi165.stdout:Total 667 kB/s | 127 kB 00:00 2024-09-17T21:44:42.831 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:42.836 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:42.836 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:42.855 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:42.899 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:42.900 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:43.033 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 639 kB/s | 80 kB 00:00 2024-09-17T21:44:43.047 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:43.053 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:43.169 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:43.184 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-fuse-19.1.1-330.gc2ddf972.el9.x86_6 3.2 MB/s | 897 kB 00:00 2024-09-17T21:44:43.184 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:43.185 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.7 MB/s | 977 kB 00:00 2024-09-17T21:44:43.186 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T21:44:43.197 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:43.221 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:43.221 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:43.259 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:43.309 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:43.309 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:43.597 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:43.634 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-17T21:44:43.712 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:43.869 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:43.968 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:43.969 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T21:44:44.106 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs2 2024-09-17T21:44:44.309 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T21:44:44.309 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:44.310 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:44.310 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T21:44:44.310 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:44.310 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2/2 2024-09-17T21:44:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/2 2024-09-17T21:44:44.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-volume 2024-09-17T21:44:44.697 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-17T21:44:44.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:44.698 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-17T21:44:44.698 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:44.698 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:44.706 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:44.798 INFO:teuthology.orchestra.run.smithi165.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:44.842 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:44.843 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:44.843 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:44.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs-devel 2024-09-17T21:44:44.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-volume 2024-09-17T21:44:45.242 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:45.381 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 285 k 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 1.3 M 2024-09-17T21:44:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:45.545 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:45.579 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:45.619 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T21:44:45.620 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:45.620 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T21:44:45.620 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:45.629 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:45.629 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:45.655 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:45.656 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:45.677 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:45.677 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 32 k 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 142 k 2024-09-17T21:44:45.678 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:45.718 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:45.719 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:45.719 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.1.1-330.gc2ddf972.el9 ceph-noarch 285 k 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:45.720 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:45.721 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 285 k 2024-09-17T21:44:45.721 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 1.3 M 2024-09-17T21:44:45.721 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:45.826 INFO:teuthology.orchestra.run.smithi165.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 215 kB/s | 32 kB 00:00 2024-09-17T21:44:45.827 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:45.827 INFO:teuthology.orchestra.run.smithi165.stdout:Total 213 kB/s | 32 kB 00:00 2024-09-17T21:44:45.827 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:45.831 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:45.831 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:45.849 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:45.849 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:45.937 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:45.947 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-volume-19.1.1-330.gc2ddf972.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-17T21:44:45.947 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:45.948 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-17T21:44:45.948 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:45.957 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:45.957 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:45.962 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:45.979 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:45.979 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:45.980 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T21:44:45.980 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:45.984 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:45.984 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:46.048 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:46.300 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T21:44:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:46.451 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:46.680 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:46.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados2 2024-09-17T21:44:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:47.074 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:47.074 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:47.075 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:47.075 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:47.342 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 1/1 2024-09-17T21:44:47.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:47.342 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:47.342 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.1.1-330.gc2ddf972.el9.noarch 2024-09-17T21:44:47.342 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:47.343 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:47.396 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados-devel 2024-09-17T21:44:47.506 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:47.595 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:47.639 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:47.640 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:47.640 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:47.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-09-17T21:44:47.719 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librbd1 2024-09-17T21:44:48.014 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:48.151 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:48.153 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 127 k 2024-09-17T21:44:48.153 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 456 k 2024-09-17T21:44:48.153 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:48.254 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:48.296 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:48.340 INFO:teuthology.orchestra.run.smithi053.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 677 kB/s | 127 kB 00:00 2024-09-17T21:44:48.340 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:48.341 INFO:teuthology.orchestra.run.smithi053.stdout:Total 671 kB/s | 127 kB 00:00 2024-09-17T21:44:48.341 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:48.347 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:48.347 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:48.385 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:48.397 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 127 k 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 127 k 2024-09-17T21:44:48.398 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 456 k 2024-09-17T21:44:48.399 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:48.414 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:48.414 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:48.429 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:48.429 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:48.430 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:48.509 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rados 2024-09-17T21:44:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:48.620 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-19.1.1-330.gc2ddf972.el9.x86_64. 573 kB/s | 127 kB 00:00 2024-09-17T21:44:48.620 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:48.621 INFO:teuthology.orchestra.run.smithi007.stdout:Total 569 kB/s | 127 kB 00:00 2024-09-17T21:44:48.621 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:48.627 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:48.627 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:48.674 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:48.695 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:48.695 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:48.854 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:48.980 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:49.094 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:49.183 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:49.228 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:49.229 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:49.229 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:49.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rgw 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:49.366 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:49.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs2 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:49.729 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:49.894 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:49.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-09-17T21:44:49.983 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:50.026 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:50.027 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:50.027 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:50.109 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-cephfs 2024-09-17T21:44:50.232 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:50.325 INFO:teuthology.orchestra.run.smithi053.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:50.372 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:50.374 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:50.374 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:50.457 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs-devel 2024-09-17T21:44:50.577 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:50.670 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:50.692 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:50.716 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:50.717 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:50.717 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:50.780 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:50.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-09-17T21:44:50.823 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:50.824 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:50.824 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:50.910 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rbd 2024-09-17T21:44:51.064 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:51.203 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:51.203 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:51.203 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 32 k 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 142 k 2024-09-17T21:44:51.204 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:51.359 INFO:teuthology.orchestra.run.smithi053.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 204 kB/s | 32 kB 00:00 2024-09-17T21:44:51.360 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:51.360 INFO:teuthology.orchestra.run.smithi053.stdout:Total 202 kB/s | 32 kB 00:00 2024-09-17T21:44:51.360 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:51.364 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:51.364 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:51.378 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:51.378 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:51.396 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:51.465 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:51.494 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:51.533 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:51.534 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:51.534 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T21:44:51.534 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:51.534 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 32 k 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 32 k 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 142 k 2024-09-17T21:44:51.535 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:51.576 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:51.584 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:51.627 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:51.628 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:51.628 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-19.1.1-330.gc2ddf972.el9.x86_64 205 kB/s | 32 kB 00:00 2024-09-17T21:44:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:51.691 INFO:teuthology.orchestra.run.smithi007.stdout:Total 203 kB/s | 32 kB 00:00 2024-09-17T21:44:51.691 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:51.695 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:51.695 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:51.712 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:51.712 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:51.715 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-fuse 2024-09-17T21:44:51.797 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:51.915 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:51.952 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:52.198 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:52.198 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:52.199 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:52.199 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:52.199 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:52.295 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:52.316 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:52.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados2 2024-09-17T21:44:52.428 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:52.429 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 85 k 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 231 k 2024-09-17T21:44:52.430 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:52.553 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:52.554 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:52.629 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 429 kB/s | 85 kB 00:00 2024-09-17T21:44:52.630 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:52.630 INFO:teuthology.orchestra.run.smithi165.stdout:Total 426 kB/s | 85 kB 00:00 2024-09-17T21:44:52.630 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:52.637 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:52.637 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:52.699 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:52.700 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:52.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-09-17T21:44:52.860 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:52.960 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:53.017 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:53.109 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:53.155 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:53.156 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:53.156 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:53.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librbd1 2024-09-17T21:44:53.326 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:53.341 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:53.418 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:53.464 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:53.465 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:53.466 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:53.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-09-17T21:44:53.571 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:53.572 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:53.572 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:53.572 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:53.572 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:53.572 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:53.833 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:53.892 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-mirror 2024-09-17T21:44:53.925 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:53.971 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:53.972 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:53.972 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:54.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rados 2024-09-17T21:44:54.149 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:54.245 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:54.291 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:54.292 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:54.292 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:54.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-09-17T21:44:54.504 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:54.645 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:54.646 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:54.646 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 3.1 M 2024-09-17T21:44:54.647 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 13 M 2024-09-17T21:44:54.648 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:54.739 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:54.787 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:54.788 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:54.788 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:54.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rgw 2024-09-17T21:44:54.968 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:54.992 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-17T21:44:54.993 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:54.994 INFO:teuthology.orchestra.run.smithi165.stdout:Total 8.9 MB/s | 3.1 MB 00:00 2024-09-17T21:44:54.994 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:55.002 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:55.002 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:55.061 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:55.084 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:55.084 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:55.107 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:55.108 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:55.108 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:55.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-09-17T21:44:55.468 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:45 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:55.523 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:55.563 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:55.585 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:55.609 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:55.610 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:55.610 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:55.610 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:55.611 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:44:55.611 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T21:44:55.611 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:44:55.611 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:44:55.611 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:55.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-cephfs 2024-09-17T21:44:55.782 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:55.875 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:55.920 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:55.922 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:55.922 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:55.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-09-17T21:44:56.296 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:56.388 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:56.434 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:56.435 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:56.435 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:56.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rbd 2024-09-17T21:44:56.599 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:56.692 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:56.738 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:56.739 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:56.739 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:56.785 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:56.785 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:56.786 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:56.786 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:56.786 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:56.786 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:56.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-09-17T21:44:57.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-nbd 2024-09-17T21:44:57.118 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:57.211 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:57.258 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:57.259 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:44:57.259 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:57.327 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-fuse 2024-09-17T21:44:57.424 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:57.515 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:57.561 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:57.562 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:44:57.562 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:57.628 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:48 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:57.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-09-17T21:44:57.762 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:57.763 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:57.763 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-09-17T21:44:57.763 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:57.764 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-09-17T21:44:57.764 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T21:44:57.764 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:57.764 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-09-17T21:44:57.764 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-09-17T21:44:57.765 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-09-17T21:44:57.765 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:57.765 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 170 k 2024-09-17T21:44:57.765 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 494 k 2024-09-17T21:44:57.765 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-09-17T21:44:57.927 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:44:57.980 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 788 kB/s | 170 kB 00:00 2024-09-17T21:44:57.981 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:57.981 INFO:teuthology.orchestra.run.smithi165.stdout:Total 781 kB/s | 170 kB 00:00 2024-09-17T21:44:57.981 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-09-17T21:44:57.989 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-09-17T21:44:57.989 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-09-17T21:44:58.052 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-09-17T21:44:58.052 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-09-17T21:44:58.066 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:44:58.066 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:44:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:58.068 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 85 k 2024-09-17T21:44:58.068 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 231 k 2024-09-17T21:44:58.068 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:44:58.206 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-09-17T21:44:58.236 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:49 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:58.254 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 458 kB/s | 85 kB 00:00 2024-09-17T21:44:58.255 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:58.255 INFO:teuthology.orchestra.run.smithi053.stdout:Total 454 kB/s | 85 kB 00:00 2024-09-17T21:44:58.255 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:44:58.263 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:44:58.263 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:44:58.310 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.330 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:44:58.330 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 85 k 2024-09-17T21:44:58.374 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 85 k 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 231 k 2024-09-17T21:44:58.375 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:44:58.482 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:44:58.568 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-19.1.1-330.gc2ddf972.el9.x86_64.rpm 443 kB/s | 85 kB 00:00 2024-09-17T21:44:58.569 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:44:58.569 INFO:teuthology.orchestra.run.smithi007.stdout:Total 440 kB/s | 85 kB 00:00 2024-09-17T21:44:58.569 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:44:58.577 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:44:58.577 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:44:58.580 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.644 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:44:58.644 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:44:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.805 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:44:58.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.961 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.986 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:58.987 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-09-17T21:44:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:58.987 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:44:58.987 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:44:59.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-nbd 2024-09-17T21:44:59.249 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:59.249 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:44:59.250 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:59.250 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:44:59.250 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:44:59.295 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:59.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-mirror 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:44:59.552 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:44:59.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-09-17T21:44:59.809 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:44:59.901 INFO:teuthology.orchestra.run.smithi165.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:44:59.944 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-09-17T21:44:59.945 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-09-17T21:44:59.945 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-09-17T21:45:00.030 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:00.151 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:50 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:45:00.292 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.1 M 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 13 M 2024-09-17T21:45:00.293 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:45:00.398 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:51 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:45:00.539 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 3.1 M 2024-09-17T21:45:00.540 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 3.1 M 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 13 M 2024-09-17T21:45:00.541 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:45:00.649 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 8.7 MB/s | 3.1 MB 00:00 2024-09-17T21:45:00.650 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:45:00.650 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.7 MB/s | 3.1 MB 00:00 2024-09-17T21:45:00.650 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:45:00.660 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:45:00.660 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:45:00.747 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:45:00.747 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:45:00.889 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-19.1.1-330.gc2ddf972.el9.x86_64.rpm 8.9 MB/s | 3.1 MB 00:00 2024-09-17T21:45:00.890 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:45:00.890 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.9 MB/s | 3.1 MB 00:00 2024-09-17T21:45:00.890 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:45:00.900 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:45:00.900 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:45:00.986 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:45:00.987 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:45:01.197 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:45:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:45:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:01.437 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:45:01.640 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:01.668 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:01.669 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T21:45:01.669 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T21:45:01.669 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:45:01.669 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T21:45:01.669 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:02.482 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:45:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:45:02.482 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:02.483 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:45:02.678 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:02.679 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:45:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:45:02.679 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:02.679 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:45:02.728 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-09-17T21:45:02.948 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-09-17T21:45:03.334 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:53 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:45:03.472 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:45:03.473 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:03.473 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-17T21:45:03.473 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 170 k 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 494 k 2024-09-17T21:45:03.474 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-17T21:45:03.552 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:54 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:45:03.691 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:45:03.691 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.1.1-330.gc2ddf972.el9 ceph 170 k 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-09-17T21:45:03.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:03.693 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 170 k 2024-09-17T21:45:03.693 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 494 k 2024-09-17T21:45:03.693 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-17T21:45:03.719 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 696 kB/s | 170 kB 00:00 2024-09-17T21:45:03.720 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:45:03.720 INFO:teuthology.orchestra.run.smithi053.stdout:Total 692 kB/s | 170 kB 00:00 2024-09-17T21:45:03.720 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-17T21:45:03.729 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-17T21:45:03.729 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-17T21:45:03.796 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-17T21:45:03.797 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-17T21:45:03.902 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-19.1.1-330.gc2ddf972.el9.x86_64.rpm 815 kB/s | 170 kB 00:00 2024-09-17T21:45:03.903 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-17T21:45:03.903 INFO:teuthology.orchestra.run.smithi007.stdout:Total 808 kB/s | 170 kB 00:00 2024-09-17T21:45:03.903 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-17T21:45:03.911 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-17T21:45:03.911 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-17T21:45:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-17T21:45:03.979 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-17T21:45:03.979 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-17T21:45:04.053 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.147 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-17T21:45:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.477 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.704 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:04.754 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:45:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 1/1 2024-09-17T21:45:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:04.952 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-17T21:45:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 2024-09-17T21:45:04.952 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:04.953 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:45:04.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-09-17T21:45:05.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-09-17T21:45:05.575 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:55 ago on Tue 17 Sep 2024 09:43:10 PM UTC. 2024-09-17T21:45:05.668 INFO:teuthology.orchestra.run.smithi053.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:45:05.714 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-17T21:45:05.715 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-17T21:45:05.715 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-17T21:45:05.790 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:05.801 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:56 ago on Tue 17 Sep 2024 09:43:09 PM UTC. 2024-09-17T21:45:05.896 INFO:teuthology.orchestra.run.smithi007.stdout:Package rbd-nbd-2:19.1.1-330.gc2ddf972.el9.x86_64 is already installed. 2024-09-17T21:45:05.942 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-17T21:45:05.943 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-17T21:45:05.943 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-17T21:45:06.010 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:06.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:45:06.154 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T21:45:06.176 INFO:teuthology.orchestra.run.smithi007.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.176 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.176 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T21:45:06.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:45:06.316 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T21:45:06.339 INFO:teuthology.orchestra.run.smithi053.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.339 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.339 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T21:45:06.341 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c2ddf9721ef254645201cdbabd9c8f49c30a586f 2024-09-17T21:45:06.481 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T21:45:06.506 INFO:teuthology.orchestra.run.smithi165.stdout:19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.506 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-330.gc2ddf972.el9 2024-09-17T21:45:06.507 INFO:teuthology.task.install:The correct ceph version 19.1.1-330.gc2ddf972 is installed. 2024-09-17T21:45:06.508 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T21:45:06.509 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:06.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:45:06.537 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:06.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:45:06.564 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:06.565 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T21:45:06.595 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T21:45:06.596 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:06.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:45:06.623 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:45:06.690 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:06.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:45:06.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:45:06.782 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:06.782 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T21:45:06.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T21:45:06.881 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T21:45:06.882 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:06.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:45:06.909 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:45:06.977 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:06.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:45:07.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:45:07.069 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:07.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T21:45:07.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T21:45:07.168 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T21:45:07.168 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:07.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:45:07.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:45:07.267 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:07.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:45:07.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:45:07.360 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:07.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T21:45:07.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T21:45:07.459 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T21:45:07.537 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T21:45:07.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-17T21:45:07.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-09-17T21:45:07.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-09-17T21:45:07.571 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T21:45:07.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:45:07.611 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:45:07.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T21:45:07.641 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T21:45:07.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:45:07.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:45:07.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T21:45:07.719 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T21:45:07.720 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T21:45:07.720 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503', 'branch': 'squid', 'description': 'rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909503', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'osd.0', 'osd.1', 'osd.2'], ['mon.b', 'mgr.x', 'osd.3', 'osd.4', 'osd.5'], ['client.0']], 'seed': 1983, 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f', 'sleep_before_teardown': 0, 'subset': '79/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b0d356b98e08f6bc59ef15652f0d0e4298fc83c7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGX8AOqRTFSCaQ6E/IziRJ+TRam/+XtROAUO3Pjg2w7PcZP9QBOvlFLrOI+PXjgSGKfyfwrx+Gl6lmZfxWcuQu0=', 'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFrxYT9MOB+pNzlpfVQR2VNbiddNqPlw+XYA8wUVZPyE7sS1PBKhSuTQoNuMQNmcUvrzzYvQa5Wvtj4S9aYUpWg=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIS20mlmssYCVZ0CQnBkLNX6IYCWDqBZQm+i6aGk7VbqMxL9f5YLDnqd+aDdyGI6pO1kfygqgA45KdMdlg+t/pg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'holebdy': 512, 'nbd': True, 'ops': 6000, 'punch_holes': True, 'readbdy': 512, 'truncbdy': 512, 'writebdy': 512}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_21:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T21:45:07.720 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:45:07.751 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:45:07.753 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T21:45:07.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:45:07.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:45:07.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T21:45:07.838 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:07.838 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:45:07.893 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:45:07.893 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-17T21:45:07.950 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 21:45:01.953692784 +0000 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 21:42:29.645672950 +0000 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 21:42:29.645672950 +0000 2024-09-17T21:45:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 21:42:29.645672950 +0000 2024-09-17T21:45:07.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:45:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T21:45:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T21:45:08.017 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000110118 s, 4.6 MB/s 2024-09-17T21:45:08.019 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:45:08.078 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-17T21:45:08.135 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:45:08.135 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 21:45:01.954692770 +0000 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 21:42:29.969668349 +0000 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 21:42:29.969668349 +0000 2024-09-17T21:45:08.136 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 21:42:29.969668349 +0000 2024-09-17T21:45:08.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:45:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T21:45:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T21:45:08.201 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000104056 s, 4.9 MB/s 2024-09-17T21:45:08.202 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:45:08.260 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-17T21:45:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:45:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 21:45:01.954692770 +0000 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 21:42:30.289663806 +0000 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 21:42:30.289663806 +0000 2024-09-17T21:45:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 21:42:30.289663806 +0000 2024-09-17T21:45:08.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:45:08.382 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T21:45:08.383 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.00010165 s, 5.0 MB/s 2024-09-17T21:45:08.384 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:45:08.441 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-17T21:45:08.497 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-17 21:45:01.955692755 +0000 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-17 21:42:30.609659263 +0000 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-17 21:42:30.609659263 +0000 2024-09-17T21:45:08.498 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-17 21:42:30.609659263 +0000 2024-09-17T21:45:08.498 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:45:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-17T21:45:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-17T21:45:08.563 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.7351e-05 s, 5.9 MB/s 2024-09-17T21:45:08.564 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:45:08.621 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:08.621 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:08.622 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T21:45:08.642 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T21:45:08.642 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:08.701 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-17 21:45:01.564162336 +0000 2024-09-17T21:45:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-17 21:42:29.864138418 +0000 2024-09-17T21:45:08.702 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-17 21:42:29.864138418 +0000 2024-09-17T21:45:08.702 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-17 21:42:29.864138418 +0000 2024-09-17T21:45:08.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T21:45:08.770 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-17T21:45:08.770 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-17T21:45:08.770 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.00012058 s, 4.2 MB/s 2024-09-17T21:45:08.772 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T21:45:08.832 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-17T21:45:08.889 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T21:45:08.889 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:08.889 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-17 21:45:01.564162336 +0000 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-17 21:42:30.179132072 +0000 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-17 21:42:30.179132072 +0000 2024-09-17T21:45:08.890 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-17 21:42:30.179132072 +0000 2024-09-17T21:45:08.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T21:45:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-17T21:45:08.956 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-17T21:45:08.957 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000115955 s, 4.4 MB/s 2024-09-17T21:45:08.958 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T21:45:09.016 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-17 21:45:01.565162316 +0000 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-17 21:42:30.493125746 +0000 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-17 21:42:30.493125746 +0000 2024-09-17T21:45:09.074 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-17 21:42:30.493125746 +0000 2024-09-17T21:45:09.075 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T21:45:09.141 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-17T21:45:09.145 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-17T21:45:09.145 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000121038 s, 4.2 MB/s 2024-09-17T21:45:09.146 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T21:45:09.204 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-17T21:45:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T21:45:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T21:45:09.262 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-17T21:45:09.262 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T21:45:09.262 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T21:45:09.263 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-17 21:45:01.565162316 +0000 2024-09-17T21:45:09.263 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-17 21:42:30.817119218 +0000 2024-09-17T21:45:09.263 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-17 21:42:30.817119218 +0000 2024-09-17T21:45:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-17 21:42:30.817119218 +0000 2024-09-17T21:45:09.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T21:45:09.329 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-17T21:45:09.329 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-17T21:45:09.329 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000110703 s, 4.6 MB/s 2024-09-17T21:45:09.330 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T21:45:09.389 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:09.389 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T21:45:09.390 INFO:tasks.ceph:Generating config... 2024-09-17T21:45:09.391 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-17T21:45:09.391 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-17T21:45:09.391 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-17T21:45:09.391 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T21:45:09.391 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T21:45:09.391 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T21:45:09.391 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T21:45:09.391 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T21:45:09.392 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-17T21:45:09.393 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T21:45:09.394 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-17T21:45:09.394 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T21:45:09.394 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T21:45:09.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T21:45:09.447 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T21:45:09.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T21:45:09.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:45:09.526 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.c', '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]'), ('mon.b', '172.21.15.53')] 2024-09-17T21:45:09.527 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.53', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-17T21:45:09.527 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:09.527 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:45:09.583 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --addv c '[v2:172.21.15.7:3301,v1:172.21.15.7:6790]' --add b 172.21.15.53 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:09.671 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:09.671 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-17T21:45:09.670070+0000 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-17T21:45:09.670070+0000 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.b 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.7:3301/0,v1:172.21.15.7:6790/0] mon.c 2024-09-17T21:45:09.672 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T21:45:09.674 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T21:45:09.689 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 195a3125-addb-4952-bbe0-175c3f942cb7... 2024-09-17T21:45:09.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:45:09.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:45:09.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T21:45:09.773 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-17T21:45:09.774 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.775 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.776 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.53" 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on pool no app = False 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 5000 2024-09-17T21:45:09.777 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-17T21:45:09.778 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.779 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-17T21:45:09.780 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:45:09.781 INFO:teuthology.orchestra.run.smithi007.stdout: osd max markdown count = 1000 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-17T21:45:09.782 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:45:09.783 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-17T21:45:09.784 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-17T21:45:09.785 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:45:09.785 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:45:09.785 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-17T21:45:09.785 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-17T21:45:09.785 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-09-17T21:45:09.786 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-09-17T21:45:09.787 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:45:09.788 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.53" 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon client directed command retry = 5 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on pool no app = False 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: ms inject socket failures = 5000 2024-09-17T21:45:09.789 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-09-17T21:45:09.790 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.791 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = bitmap 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 1/20 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 1/20 2024-09-17T21:45:09.792 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 4/10 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: osd max markdown count = 1000 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.793 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-09-17T21:45:09.794 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:45:09.795 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-09-17T21:45:09.796 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-09-17T21:45:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T21:45:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-09-17T21:45:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-09-17T21:45:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.53" 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: mon client directed command retry = 5 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on pool no app = False 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: ms inject socket failures = 5000 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-09-17T21:45:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug trim objects = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bdev async discard = True 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bdev enable discard = True 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore allocator = bitmap 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-09-17T21:45:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: osd blocked scrub grace period = 3600 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: osd max markdown count = 1000 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.804 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-09-17T21:45:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T21:45:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-09-17T21:45:09.807 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-09-17T21:45:09.809 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T21:45:09.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T21:45:09.904 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T21:45:09.904 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:09.904 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T21:45:09.920 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:09.920 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T21:45:09.976 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:45:09.976 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:09.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:45:09.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:45:10.052 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:10.052 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:10.109 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:45:10.109 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:10.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:45:10.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:45:10.154 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:10.154 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:10.212 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-09-17T21:45:10.212 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:10.212 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T21:45:10.212 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T21:45:10.253 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:10.253 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:10.311 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T21:45:10.311 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T21:45:10.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:45:10.411 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T21:45:10.414 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T21:45:10.414 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T21:45:10.415 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T21:45:10.471 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T21:45:10.484 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T21:45:10.485 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-09-17T21:45:10.485 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T21:45:10.512 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:10.512 INFO:tasks.ceph:role: osd.0 2024-09-17T21:45:10.512 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:45:10.512 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:45:10.578 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:10.578 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:10.578 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:10.579 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:10.610 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T21:45:10.612 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T21:45:10.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T21:45:10.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T21:45:10.756 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T21:45:10.823 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:10.823 INFO:tasks.ceph:role: osd.1 2024-09-17T21:45:10.823 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:45:10.823 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:45:10.888 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:10.888 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:10.888 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:10.889 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:10.921 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T21:45:10.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T21:45:10.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T21:45:10.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T21:45:11.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T21:45:11.129 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:11.129 INFO:tasks.ceph:role: osd.2 2024-09-17T21:45:11.129 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:45:11.129 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:45:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:11.195 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:11.226 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-17T21:45:11.229 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-17T21:45:11.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T21:45:11.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T21:45:11.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:11.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:11.469+0000 7f2feaee2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T21:45:11.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:11.469+0000 7f2feaee2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T21:45:11.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:11.469+0000 7f2feaee2640 -1 bdev(0x555944227800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:11.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:11.469+0000 7f2feaee2640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T21:45:14.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T21:45:14.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:15.010+0000 7fb587618640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T21:45:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:15.010+0000 7fb587618640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T21:45:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:15.010+0000 7fb587618640 -1 bdev(0x55ec5ba15800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:15.010+0000 7fb587618640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T21:45:18.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T21:45:18.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:18.538+0000 7fc21de1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T21:45:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:18.538+0000 7fc21de1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T21:45:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:18.538+0000 7fc21de1b640 -1 bdev(0x560696ab7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-17T21:45:18.538+0000 7fc21de1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T21:45:21.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T21:45:21.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T21:45:21.762 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:21.762 INFO:tasks.ceph:role: osd.3 2024-09-17T21:45:21.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:45:21.763 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T21:45:21.832 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:21.833 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:21.864 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-17T21:45:21.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-17T21:45:21.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-09-17T21:45:21.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T21:45:22.011 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T21:45:22.078 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:22.079 INFO:tasks.ceph:role: osd.4 2024-09-17T21:45:22.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:45:22.079 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T21:45:22.144 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:22.145 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:22.176 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-17T21:45:22.178 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-17T21:45:22.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-09-17T21:45:22.248 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T21:45:22.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T21:45:22.385 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-09-17T21:45:22.385 INFO:tasks.ceph:role: osd.5 2024-09-17T21:45:22.385 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:45:22.385 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-17T21:45:22.451 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T21:45:22.452 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T21:45:22.452 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T21:45:22.452 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T21:45:22.485 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-17T21:45:22.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-17T21:45:22.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-09-17T21:45:22.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T21:45:22.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:22.742+0000 7f16fbb43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T21:45:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:22.742+0000 7f16fbb43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T21:45:22.745 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:22.742+0000 7f16fbb43640 -1 bdev(0x55bf4f3ab800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:22.746 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:22.742+0000 7f16fbb43640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T21:45:25.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T21:45:25.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:25.793+0000 7f2e31e16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T21:45:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:25.793+0000 7f2e31e16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T21:45:25.796 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:25.793+0000 7f2e31e16640 -1 bdev(0x56480168f800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:25.797 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:25.794+0000 7f2e31e16640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T21:45:28.731 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T21:45:28.763 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:28.852+0000 7fbf69a9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T21:45:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:28.852+0000 7fbf69a9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T21:45:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:28.852+0000 7fbf69a9c640 -1 bdev(0x55a9544ff800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T21:45:28.855 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-17T21:45:28.852+0000 7fbf69a9c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T21:45:32.014 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T21:45:32.045 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T21:45:32.045 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:32.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T21:45:32.077 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:32.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T21:45:32.148 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:32.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T21:45:32.219 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:32.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T21:45:32.250 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:32.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T21:45:32.320 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:32.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T21:45:32.389 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:32.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T21:45:32.458 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-09-17T21:45:32.459 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T21:45:32.478 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T21:45:32.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:45:32.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBn+OlmUzcDHBAA95vyii8PG0gXGjlKykH4uA== 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBr+OlmqyeoABAABjXB5zrXiF5QuLnwjOnrkA== 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBu+OlmIGMhIBAAn/D0q12wWSUYUxRnPMuVcw== 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBm+Olm0sBxGBAAznyT0RdGDNYE9NpEgPj30A== 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBy+Olm0zFTLBAALPxA3MrjYsqNtv9BJw+geA== 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-17T21:45:32.508 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB1+Olm4WZeLxAA/rBMkk1+nEoBcF45nkO1Xw== 2024-09-17T21:45:32.509 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-17T21:45:32.509 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQB4+Olm6qfaMhAASiepkvGkVIj0qkjWjwOUDg== 2024-09-17T21:45:32.509 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-17T21:45:32.509 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBm+Olmxe77GxAAIur3dRXPBHISLxgBiNgitQ== 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBn+OlmUzcDHBAA95vyii8PG0gXGjlKykH4uA== 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBr+OlmqyeoABAABjXB5zrXiF5QuLnwjOnrkA== 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBu+OlmIGMhIBAAn/D0q12wWSUYUxRnPMuVcw== 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-09-17T21:45:32.525 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBm+Olm0sBxGBAAznyT0RdGDNYE9NpEgPj30A== 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBy+Olm0zFTLBAALPxA3MrjYsqNtv9BJw+geA== 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQB1+Olm4WZeLxAA/rBMkk1+nEoBcF45nkO1Xw== 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQB4+Olm6qfaMhAASiepkvGkVIj0qkjWjwOUDg== 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-09-17T21:45:32.526 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQBm+Olmxe77GxAAIur3dRXPBHISLxgBiNgitQ== 2024-09-17T21:45:32.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:45:32.736 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T21:45:32.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.900 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T21:45:32.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:45:32.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T21:45:33.012 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T21:45:33.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T21:45:33.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:45:33.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T21:45:33.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T21:45:33.228 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:45:33.325 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T21:45:33.351 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T21:45:33.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T21:45:33.475 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T21:45:33.501 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:33.504 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T21:45:33.557 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T21:45:33.558 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T21:45:33.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T21:45:33.560 INFO:tasks.ceph.mon.a:Started 2024-09-17T21:45:33.560 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T21:45:33.561 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T21:45:33.563 INFO:tasks.ceph.mon.c:Started 2024-09-17T21:45:33.563 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T21:45:33.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T21:45:33.600 INFO:tasks.ceph.mon.b:Started 2024-09-17T21:45:33.600 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T21:45:33.600 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T21:45:33.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T21:45:33.602 INFO:tasks.ceph.mgr.x:Started 2024-09-17T21:45:33.602 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T21:45:33.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T21:45:33.939 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-17T21:45:33.950 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T21:45:33.950 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T21:45:33.951 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:33.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T21:45:33.968 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:33.965+0000 7f1265c0d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-17T21:45:33.976 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:33.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T21:45:34.010 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.008+0000 7f1265c0d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-17T21:45:34.043 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-17T21:45:34.043 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T21:45:34.108 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:34.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T21:45:34.134 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:34.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T21:45:34.172 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.169+0000 7f1265c0d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-17T21:45:34.197 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-17T21:45:34.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T21:45:34.226 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.223+0000 7f1265c0d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-17T21:45:34.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 2e180d1f-2ad9-4830-8725-c22ba929de39 0 2024-09-17T21:45:34.405 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.403+0000 7f1265c0d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-17T21:45:34.477 INFO:teuthology.orchestra.run.smithi053.stdout:0 2024-09-17T21:45:34.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new 4d534322-006a-4257-b243-601aa8088f95 1 2024-09-17T21:45:34.524 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.522+0000 7f1265c0d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-17T21:45:34.671 INFO:teuthology.orchestra.run.smithi053.stdout:1 2024-09-17T21:45:34.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298 2 2024-09-17T21:45:34.880 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.877+0000 7f1265c0d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-17T21:45:34.906 INFO:teuthology.orchestra.run.smithi053.stdout:2 2024-09-17T21:45:34.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new f9403889-7fb8-4c49-8193-55ec6ae285cc 3 2024-09-17T21:45:34.935 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:34.932+0000 7f1265c0d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-17T21:45:35.096 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:35.093+0000 7f1265c0d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-17T21:45:35.100 INFO:teuthology.orchestra.run.smithi053.stdout:3 2024-09-17T21:45:35.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new cf4edbdc-6ec7-496f-94b6-7e6128956d63 4 2024-09-17T21:45:35.334 INFO:teuthology.orchestra.run.smithi053.stdout:4 2024-09-17T21:45:35.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd new ebca2188-456e-4d5c-956d-6382dd41b9bd 5 2024-09-17T21:45:35.529 INFO:teuthology.orchestra.run.smithi053.stdout:5 2024-09-17T21:45:35.540 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T21:45:35.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T21:45:35.543 INFO:tasks.ceph.osd.0:Started 2024-09-17T21:45:35.543 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T21:45:35.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T21:45:35.546 INFO:tasks.ceph.osd.1:Started 2024-09-17T21:45:35.546 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T21:45:35.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T21:45:35.550 INFO:tasks.ceph.osd.2:Started 2024-09-17T21:45:35.550 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T21:45:35.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T21:45:35.552 INFO:tasks.ceph.osd.3:Started 2024-09-17T21:45:35.552 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T21:45:35.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T21:45:35.557 INFO:tasks.ceph.osd.4:Started 2024-09-17T21:45:35.557 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T21:45:35.557 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T21:45:35.560 INFO:tasks.ceph.osd.5:Started 2024-09-17T21:45:35.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:35.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:35.747 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:35.525589+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:35.760 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T21:45:35.760 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T21:45:36.113 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.110+0000 7f1265c0d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-17T21:45:36.168 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.165+0000 7f1265c0d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-17T21:45:36.365 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:36.533 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-17T21:45:36.533 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":8,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:35.525589+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:36.542 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-09-17T21:45:36.553 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.550+0000 7f1265c0d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-17T21:45:36.642 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.639+0000 7f1265c0d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-17T21:45:36.753 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.750+0000 7f1265c0d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-17T21:45:36.801 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.799+0000 7f1265c0d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-17T21:45:36.920 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.917+0000 7f1265c0d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-17T21:45:36.975 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:36.973+0000 7f1265c0d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-17T21:45:37.022 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.020+0000 7f1265c0d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-17T21:45:37.139 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.137+0000 7f1265c0d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-17T21:45:37.192 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.189+0000 7f1265c0d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-17T21:45:37.245 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.243+0000 7f1265c0d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-17T21:45:37.256 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:45:37.253+0000 7f7ef3815640 -1 Falling back to public interface 2024-09-17T21:45:37.279 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:45:37.276+0000 7fc4788e0640 -1 Falling back to public interface 2024-09-17T21:45:37.316 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.313+0000 7f1265c0d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-17T21:45:37.362 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.360+0000 7f1265c0d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-17T21:45:37.408 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:37.406+0000 7f1265c0d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-17T21:45:37.507 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:45:37.504+0000 7fbd67c18640 -1 Falling back to public interface 2024-09-17T21:45:37.511 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:45:37.508+0000 7f0228aa5640 -1 Falling back to public interface 2024-09-17T21:45:37.532 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:45:37.529+0000 7fa4c812e640 -1 Falling back to public interface 2024-09-17T21:45:37.677 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:45:37.674+0000 7f43f2285640 -1 Falling back to public interface 2024-09-17T21:45:38.093 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:38.090+0000 7f1265c0d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-17T21:45:39.105 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:39.103+0000 7f1205775640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T21:45:39.106 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:45:39.103+0000 7f1205775640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T21:45:41.036 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:45:41.033+0000 7f7ef3815640 -1 osd.1 0 log_to_monitors true 2024-09-17T21:45:41.289 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:45:41.287+0000 7fbd67c18640 -1 osd.0 0 log_to_monitors true 2024-09-17T21:45:41.570 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:45:41.568+0000 7fa4c812e640 -1 osd.5 0 log_to_monitors true 2024-09-17T21:45:41.675 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:45:41.672+0000 7f43f2285640 -1 osd.3 0 log_to_monitors true 2024-09-17T21:45:41.765 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:45:41.762+0000 7f0228aa5640 -1 osd.4 0 log_to_monitors true 2024-09-17T21:45:41.822 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:45:41.820+0000 7fc4788e0640 -1 osd.2 0 log_to_monitors true 2024-09-17T21:45:42.841 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:45:42.838+0000 7f7eef93a640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T21:45:42.857 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:45:42.855+0000 7f7ee6f35640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:43.148 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:43.523 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-17T21:45:43.523 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":11,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:43.109362+0000","last_up_change":"2024-09-17T21:45:43.109362+0000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:43.538 DEBUG:teuthology.misc:1 of 6 OSDs are up 2024-09-17T21:45:44.128 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:45:44.126+0000 7fbd62d77640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T21:45:44.141 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:45:44.139+0000 7fbd5b3a9640 -1 osd.0 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:44.166 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:45:44.164+0000 7fc47404c640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T21:45:44.181 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:45:44.179+0000 7fc46c67e640 -1 osd.2 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:44.283 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:45:44.281+0000 7f43ed9f1640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T21:45:44.293 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:45:44.291+0000 7f43e6023640 -1 osd.3 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:44.305 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:45:44.303+0000 7fa4c48d1640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T21:45:44.316 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:45:44.314+0000 7fa4bbecc640 -1 osd.5 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:44.349 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:45:44.346+0000 7f0224211640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T21:45:44.362 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:45:44.360+0000 7f021c843640 -1 osd.4 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:45:50.144 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:50.438 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-17T21:45:50.439 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":15,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:48.155064+0000","last_up_change":"2024-09-17T21:45:45.117729+0000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6817","nonce":2260220328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6819","nonce":2260220328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6823","nonce":2260220328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6821","nonce":2260220328}]},"public_addr":"172.21.15.53:6817/2260220328","cluster_addr":"172.21.15.53:6819/2260220328","heartbeat_back_addr":"172.21.15.53:6823/2260220328","heartbeat_front_addr":"172.21.15.53:6821/2260220328","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:50.450 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-09-17T21:45:50.450 INFO:tasks.ceph:Creating RBD pool 2024-09-17T21:45:50.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-17T21:45:51.229 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2024-09-17T21:45:51.244 DEBUG:teuthology.orchestra.run.smithi007:> rbd --cluster ceph pool init rbd 2024-09-17T21:45:54.250 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T21:45:54.251 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T21:45:54.251 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T21:45:54.251 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T21:45:54.251 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T21:45:54.598 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:54.610 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":14223622},{"type":"v1","addr":"172.21.15.53:6825","nonce":14223622}]},"active_addr":"172.21.15.53:6825/14223622","active_change":"2024-09-17T21:45:38.093216+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":607057819}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":2976419226}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":911504094}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.53:0","nonce":729917129}]}]} 2024-09-17T21:45:54.612 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T21:45:54.612 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:45:54.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:54.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:54.902 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:54.234200+0000","last_up_change":"2024-09-17T21:45:45.117729+0000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:45:50.763887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6817","nonce":2260220328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6819","nonce":2260220328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6823","nonce":2260220328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6821","nonce":2260220328}]},"public_addr":"172.21.15.53:6817/2260220328","cluster_addr":"172.21.15.53:6819/2260220328","heartbeat_back_addr":"172.21.15.53:6823/2260220328","heartbeat_front_addr":"172.21.15.53:6821/2260220328","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:54.915 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:45:54.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:45:55.204 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:55.205 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:45:54.234200+0000","last_up_change":"2024-09-17T21:45:45.117729+0000","last_in_change":"2024-09-17T21:45:35.525589+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:45:50.763887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6817","nonce":2260220328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6819","nonce":2260220328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6823","nonce":2260220328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":2260220328},{"type":"v1","addr":"172.21.15.53:6821","nonce":2260220328}]},"public_addr":"172.21.15.53:6817/2260220328","cluster_addr":"172.21.15.53:6819/2260220328","heartbeat_back_addr":"172.21.15.53:6823/2260220328","heartbeat_front_addr":"172.21.15.53:6821/2260220328","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:45:55.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:45:55.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:45:55.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:45:55.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:45:55.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:45:55.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:45:55.376 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:55.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:45:55.382 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:55.383 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:45:55.425 INFO:teuthology.orchestra.run.smithi007.stdout:51539607557 2024-09-17T21:45:55.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:45:55.426 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-09-17T21:45:55.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:45:55.438 INFO:teuthology.orchestra.run.smithi007.stdout:51539607557 2024-09-17T21:45:55.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:45:55.440 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:55.441 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:45:55.762 INFO:teuthology.orchestra.run.smithi007.stdout:51539607555 2024-09-17T21:45:55.774 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:55.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.0 2024-09-17T21:45:55.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607556 for osd.5 2024-09-17T21:45:55.875 INFO:teuthology.orchestra.run.smithi007.stdout:51539607555 2024-09-17T21:45:55.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.3 2024-09-17T21:45:55.898 INFO:teuthology.orchestra.run.smithi007.stdout:47244640259 2024-09-17T21:45:55.910 INFO:teuthology.orchestra.run.smithi007.stdout:51539607555 2024-09-17T21:45:55.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.1 2024-09-17T21:45:55.920 INFO:teuthology.orchestra.run.smithi007.stdout:51539607555 2024-09-17T21:45:55.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607555 for osd.2 2024-09-17T21:45:55.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607555 for osd.4 2024-09-17T21:45:56.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:45:56.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:45:56.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:45:56.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:45:56.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:45:56.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:45:57.165 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:57.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.0 2024-09-17T21:45:57.183 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.243 INFO:teuthology.orchestra.run.smithi007.stdout:51539607557 2024-09-17T21:45:57.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.5 2024-09-17T21:45:57.263 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.279 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:57.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.2 2024-09-17T21:45:57.293 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.337 INFO:teuthology.orchestra.run.smithi007.stdout:51539607556 2024-09-17T21:45:57.348 INFO:teuthology.orchestra.run.smithi007.stdout:51539607557 2024-09-17T21:45:57.350 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607556 got 51539607556 for osd.3 2024-09-17T21:45:57.350 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-09-17T21:45:57.350 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607557 got 51539607557 for osd.4 2024-09-17T21:45:57.362 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2024-09-17T21:45:57.363 DEBUG:teuthology.parallel:result is None 2024-09-17T21:45:57.363 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:45:57.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:45:57.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:57.689 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:45:57.701 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-17T21:45:56.101109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201848,"kb_used_data":2212,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029192,"statfs":{"total":579820584960,"available":579613892608,"internally_reserved":0,"allocated":2265088,"data_stored":1532674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":204266144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875296"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626245+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:45:54.626245+0000","last_peered":"2024-09-17T21:45:54.626245+0000","last_clean":"2024-09-17T21:45:54.626245+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:45:54.626245+0000","last_undegraded":"2024-09-17T21:45:54.626245+0000","last_fullsized":"2024-09-17T21:45:54.626245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240609+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:45:54.240609+0000","last_peered":"2024-09-17T21:45:54.240609+0000","last_clean":"2024-09-17T21:45:54.240609+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:45:54.240609+0000","last_undegraded":"2024-09-17T21:45:54.240609+0000","last_fullsized":"2024-09-17T21:45:54.240609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626592+0000","last_change":"2024-09-17T21:45:54.626592+0000","last_active":"2024-09-17T21:45:54.626592+0000","last_peered":"2024-09-17T21:45:54.626592+0000","last_clean":"2024-09-17T21:45:54.626592+0000","last_became_active":"2024-09-17T21:45:52.233689+0000","last_became_peered":"2024-09-17T21:45:52.233689+0000","last_unstale":"2024-09-17T21:45:54.626592+0000","last_undegraded":"2024-09-17T21:45:54.626592+0000","last_fullsized":"2024-09-17T21:45:54.626592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:26:32.461807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.241161+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236800+0000","last_became_peered":"2024-09-17T21:45:52.236800+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:01.477198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.243580+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:45:54.243580+0000","last_peered":"2024-09-17T21:45:54.243580+0000","last_clean":"2024-09-17T21:45:54.243580+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:45:54.243580+0000","last_undegraded":"2024-09-17T21:45:54.243580+0000","last_fullsized":"2024-09-17T21:45:54.243580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.241342+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:45:54.241342+0000","last_peered":"2024-09-17T21:45:54.241342+0000","last_clean":"2024-09-17T21:45:54.241342+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:45:54.241342+0000","last_undegraded":"2024-09-17T21:45:54.241342+0000","last_fullsized":"2024-09-17T21:45:54.241342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626274+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:45:54.626274+0000","last_peered":"2024-09-17T21:45:54.626274+0000","last_clean":"2024-09-17T21:45:54.626274+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:45:54.626274+0000","last_undegraded":"2024-09-17T21:45:54.626274+0000","last_fullsized":"2024-09-17T21:45:54.626274+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.625689+0000","last_change":"2024-09-17T21:45:47.256280+0000","last_active":"2024-09-17T21:45:54.625689+0000","last_peered":"2024-09-17T21:45:54.625689+0000","last_clean":"2024-09-17T21:45:54.625689+0000","last_became_active":"2024-09-17T21:45:47.255320+0000","last_became_peered":"2024-09-17T21:45:47.255320+0000","last_unstale":"2024-09-17T21:45:54.625689+0000","last_undegraded":"2024-09-17T21:45:54.625689+0000","last_fullsized":"2024-09-17T21:45:54.625689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:18.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":761856,"data_stored":644878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336304,"statfs":{"total":96636764160,"available":96600375296,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:45:57.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:45:57.987 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:57.988 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:45:58.000 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-17T21:45:56.101109+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201848,"kb_used_data":2212,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029192,"statfs":{"total":579820584960,"available":579613892608,"internally_reserved":0,"allocated":2265088,"data_stored":1532674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":204266144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.875296"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626245+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:45:54.626245+0000","last_peered":"2024-09-17T21:45:54.626245+0000","last_clean":"2024-09-17T21:45:54.626245+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:45:54.626245+0000","last_undegraded":"2024-09-17T21:45:54.626245+0000","last_fullsized":"2024-09-17T21:45:54.626245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240609+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:45:54.240609+0000","last_peered":"2024-09-17T21:45:54.240609+0000","last_clean":"2024-09-17T21:45:54.240609+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:45:54.240609+0000","last_undegraded":"2024-09-17T21:45:54.240609+0000","last_fullsized":"2024-09-17T21:45:54.240609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626592+0000","last_change":"2024-09-17T21:45:54.626592+0000","last_active":"2024-09-17T21:45:54.626592+0000","last_peered":"2024-09-17T21:45:54.626592+0000","last_clean":"2024-09-17T21:45:54.626592+0000","last_became_active":"2024-09-17T21:45:52.233689+0000","last_became_peered":"2024-09-17T21:45:52.233689+0000","last_unstale":"2024-09-17T21:45:54.626592+0000","last_undegraded":"2024-09-17T21:45:54.626592+0000","last_fullsized":"2024-09-17T21:45:54.626592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:26:32.461807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.241161+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236800+0000","last_became_peered":"2024-09-17T21:45:52.236800+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:01.477198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.243580+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:45:54.243580+0000","last_peered":"2024-09-17T21:45:54.243580+0000","last_clean":"2024-09-17T21:45:54.243580+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:45:54.243580+0000","last_undegraded":"2024-09-17T21:45:54.243580+0000","last_fullsized":"2024-09-17T21:45:54.243580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.241342+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:45:54.241342+0000","last_peered":"2024-09-17T21:45:54.241342+0000","last_clean":"2024-09-17T21:45:54.241342+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:45:54.241342+0000","last_undegraded":"2024-09-17T21:45:54.241342+0000","last_fullsized":"2024-09-17T21:45:54.241342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626274+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:45:54.626274+0000","last_peered":"2024-09-17T21:45:54.626274+0000","last_clean":"2024-09-17T21:45:54.626274+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:45:54.626274+0000","last_undegraded":"2024-09-17T21:45:54.626274+0000","last_fullsized":"2024-09-17T21:45:54.626274+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.625689+0000","last_change":"2024-09-17T21:45:47.256280+0000","last_active":"2024-09-17T21:45:54.625689+0000","last_peered":"2024-09-17T21:45:54.625689+0000","last_clean":"2024-09-17T21:45:54.625689+0000","last_became_active":"2024-09-17T21:45:47.255320+0000","last_became_peered":"2024-09-17T21:45:47.255320+0000","last_unstale":"2024-09-17T21:45:54.625689+0000","last_undegraded":"2024-09-17T21:45:54.625689+0000","last_fullsized":"2024-09-17T21:45:54.625689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:18.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607556,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31936,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339904,"statfs":{"total":96636764160,"available":96604061696,"internally_reserved":0,"allocated":761856,"data_stored":644878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607556,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607556,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336304,"statfs":{"total":96636764160,"available":96600375296,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:45:58.000 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:45:58.001 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T21:45:58.001 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T21:45:58.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T21:45:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:58.324 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T21:45:58.336 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T21:45:58.337 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-17T21:45:58.348 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-17T21:45:58.348 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub'], 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'extra_packages': ['rbd-nbd'], 'flavor': 'default', 'sha1': 'c2ddf9721ef254645201cdbabd9c8f49c30a586f'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b0d356b98e08f6bc59ef15652f0d0e4298fc83c7'}} 2024-09-17T21:45:58.349 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-17T21:45:58.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-17T21:45:58.657 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-17T21:45:58.658 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:45:58.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:45:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:58.948 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:45:58.960 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T21:45:58.646514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201864,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029176,"statfs":{"total":579820584960,"available":579613876224,"internally_reserved":0,"allocated":2277376,"data_stored":1537642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":204266144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416765"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626245+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:45:54.626245+0000","last_peered":"2024-09-17T21:45:54.626245+0000","last_clean":"2024-09-17T21:45:54.626245+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:45:54.626245+0000","last_undegraded":"2024-09-17T21:45:54.626245+0000","last_fullsized":"2024-09-17T21:45:54.626245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240609+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:45:54.240609+0000","last_peered":"2024-09-17T21:45:54.240609+0000","last_clean":"2024-09-17T21:45:54.240609+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:45:54.240609+0000","last_undegraded":"2024-09-17T21:45:54.240609+0000","last_fullsized":"2024-09-17T21:45:54.240609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626592+0000","last_change":"2024-09-17T21:45:54.626592+0000","last_active":"2024-09-17T21:45:54.626592+0000","last_peered":"2024-09-17T21:45:54.626592+0000","last_clean":"2024-09-17T21:45:54.626592+0000","last_became_active":"2024-09-17T21:45:52.233689+0000","last_became_peered":"2024-09-17T21:45:52.233689+0000","last_unstale":"2024-09-17T21:45:54.626592+0000","last_undegraded":"2024-09-17T21:45:54.626592+0000","last_fullsized":"2024-09-17T21:45:54.626592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:26:32.461807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.241161+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236800+0000","last_became_peered":"2024-09-17T21:45:52.236800+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:01.477198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.243580+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:45:54.243580+0000","last_peered":"2024-09-17T21:45:54.243580+0000","last_clean":"2024-09-17T21:45:54.243580+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:45:54.243580+0000","last_undegraded":"2024-09-17T21:45:54.243580+0000","last_fullsized":"2024-09-17T21:45:54.243580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.241342+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:45:54.241342+0000","last_peered":"2024-09-17T21:45:54.241342+0000","last_clean":"2024-09-17T21:45:54.241342+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:45:54.241342+0000","last_undegraded":"2024-09-17T21:45:54.241342+0000","last_fullsized":"2024-09-17T21:45:54.241342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626274+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:45:54.626274+0000","last_peered":"2024-09-17T21:45:54.626274+0000","last_clean":"2024-09-17T21:45:54.626274+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:45:54.626274+0000","last_undegraded":"2024-09-17T21:45:54.626274+0000","last_fullsized":"2024-09-17T21:45:54.626274+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.625689+0000","last_change":"2024-09-17T21:45:47.256280+0000","last_active":"2024-09-17T21:45:54.625689+0000","last_peered":"2024-09-17T21:45:54.625689+0000","last_clean":"2024-09-17T21:45:54.625689+0000","last_became_active":"2024-09-17T21:45:47.255320+0000","last_became_peered":"2024-09-17T21:45:47.255320+0000","last_unstale":"2024-09-17T21:45:54.625689+0000","last_undegraded":"2024-09-17T21:45:54.625689+0000","last_fullsized":"2024-09-17T21:45:54.625689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:18.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336304,"statfs":{"total":96636764160,"available":96600375296,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:45:58.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:45:59.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:45:59.247 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:45:59.259 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-17T21:45:58.646514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":201864,"kb_used_data":2224,"kb_used_omap":9,"kb_used_meta":199478,"kb_avail":566029176,"statfs":{"total":579820584960,"available":579613876224,"internally_reserved":0,"allocated":2277376,"data_stored":1537642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":204266144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.416765"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626245+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:45:54.626245+0000","last_peered":"2024-09-17T21:45:54.626245+0000","last_clean":"2024-09-17T21:45:54.626245+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:45:54.626245+0000","last_undegraded":"2024-09-17T21:45:54.626245+0000","last_fullsized":"2024-09-17T21:45:54.626245+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240609+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:45:54.240609+0000","last_peered":"2024-09-17T21:45:54.240609+0000","last_clean":"2024-09-17T21:45:54.240609+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:45:54.240609+0000","last_undegraded":"2024-09-17T21:45:54.240609+0000","last_fullsized":"2024-09-17T21:45:54.240609+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626592+0000","last_change":"2024-09-17T21:45:54.626592+0000","last_active":"2024-09-17T21:45:54.626592+0000","last_peered":"2024-09-17T21:45:54.626592+0000","last_clean":"2024-09-17T21:45:54.626592+0000","last_became_active":"2024-09-17T21:45:52.233689+0000","last_became_peered":"2024-09-17T21:45:52.233689+0000","last_unstale":"2024-09-17T21:45:54.626592+0000","last_undegraded":"2024-09-17T21:45:54.626592+0000","last_fullsized":"2024-09-17T21:45:54.626592+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:26:32.461807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.241161+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236800+0000","last_became_peered":"2024-09-17T21:45:52.236800+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:44:01.477198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"19'2","reported_seq":21,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.243580+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:45:54.243580+0000","last_peered":"2024-09-17T21:45:54.243580+0000","last_clean":"2024-09-17T21:45:54.243580+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:45:54.243580+0000","last_undegraded":"2024-09-17T21:45:54.243580+0000","last_fullsized":"2024-09-17T21:45:54.243580+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.241342+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:45:54.241342+0000","last_peered":"2024-09-17T21:45:54.241342+0000","last_clean":"2024-09-17T21:45:54.241342+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:45:54.241342+0000","last_undegraded":"2024-09-17T21:45:54.241342+0000","last_fullsized":"2024-09-17T21:45:54.241342+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.626274+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:45:54.626274+0000","last_peered":"2024-09-17T21:45:54.626274+0000","last_clean":"2024-09-17T21:45:54.626274+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:45:54.626274+0000","last_undegraded":"2024-09-17T21:45:54.626274+0000","last_fullsized":"2024-09-17T21:45:54.626274+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"17'1","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.240677+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:45:54.240677+0000","last_peered":"2024-09-17T21:45:54.240677+0000","last_clean":"2024-09-17T21:45:54.240677+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:45:54.240677+0000","last_undegraded":"2024-09-17T21:45:54.240677+0000","last_fullsized":"2024-09-17T21:45:54.240677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-09-17T21:45:54.625689+0000","last_change":"2024-09-17T21:45:47.256280+0000","last_active":"2024-09-17T21:45:54.625689+0000","last_peered":"2024-09-17T21:45:54.625689+0000","last_clean":"2024-09-17T21:45:54.625689+0000","last_became_active":"2024-09-17T21:45:47.255320+0000","last_became_peered":"2024-09-17T21:45:47.255320+0000","last_unstale":"2024-09-17T21:45:54.625689+0000","last_undegraded":"2024-09-17T21:45:54.625689+0000","last_fullsized":"2024-09-17T21:45:54.625689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:32:18.349522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607558,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31372,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340468,"statfs":{"total":96636764160,"available":96604639232,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607557,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607557,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35528,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336312,"statfs":{"total":96636764160,"available":96600383488,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607557,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36108,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335732,"statfs":{"total":96636764160,"available":96599789568,"internally_reserved":0,"allocated":774144,"data_stored":649846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":180224,"data_stored":59478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336304,"statfs":{"total":96636764160,"available":96600375296,"internally_reserved":0,"allocated":184320,"data_stored":59497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:45:59.260 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:45:59.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 20 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 195a3125-addb-4952-bbe0-175c3f942cb7 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-17T21:45:33.855733+0000 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:modified 2024-09-17T21:45:58.644871+0000 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:crush_version 5 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:full_ratio 0.95 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:backfillfull_ratio 0.9 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:nearfull_ratio 0.85 2024-09-17T21:45:59.547 INFO:teuthology.orchestra.run.smithi007.stdout:require_min_compat_client luminous 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:min_compat_client jewel 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:require_osd_release squid 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:stretch_mode_enabled false 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 15 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 6.06 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 19 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.25 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:max_osd 6 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/3113291974,v1:172.21.15.7:6817/3113291974] [v2:172.21.15.7:6818/3113291974,v1:172.21.15.7:6819/3113291974] exists,up 2e180d1f-2ad9-4830-8725-c22ba929de39 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/2863076940,v1:172.21.15.7:6801/2863076940] [v2:172.21.15.7:6802/2863076940,v1:172.21.15.7:6803/2863076940] exists,up 4d534322-006a-4257-b243-601aa8088f95 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/3631167928,v1:172.21.15.7:6809/3631167928] [v2:172.21.15.7:6810/3631167928,v1:172.21.15.7:6811/3631167928] exists,up c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/2260220328,v1:172.21.15.53:6817/2260220328] [v2:172.21.15.53:6818/2260220328,v1:172.21.15.53:6819/2260220328] exists,up f9403889-7fb8-4c49-8193-55ec6ae285cc 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/2074654185,v1:172.21.15.53:6801/2074654185] [v2:172.21.15.53:6802/2074654185,v1:172.21.15.53:6803/2074654185] exists,up cf4edbdc-6ec7-496f-94b6-7e6128956d63 2024-09-17T21:45:59.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2633163970,v1:172.21.15.53:6809/2633163970] [v2:172.21.15.53:6810/2633163970,v1:172.21.15.53:6811/2633163970] exists,up ebca2188-456e-4d5c-956d-6382dd41b9bd 2024-09-17T21:45:59.559 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6816/3113291974,v1:172.21.15.7:6817/3113291974] [v2:172.21.15.7:6818/3113291974,v1:172.21.15.7:6819/3113291974] exists,up 2e180d1f-2ad9-4830-8725-c22ba929de39', 'osd.1 up in weight 1 up_from 11 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6800/2863076940,v1:172.21.15.7:6801/2863076940] [v2:172.21.15.7:6802/2863076940,v1:172.21.15.7:6803/2863076940] exists,up 4d534322-006a-4257-b243-601aa8088f95', 'osd.2 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.7:6808/3631167928,v1:172.21.15.7:6809/3631167928] [v2:172.21.15.7:6810/3631167928,v1:172.21.15.7:6811/3631167928] exists,up c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'osd.3 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/2260220328,v1:172.21.15.53:6817/2260220328] [v2:172.21.15.53:6818/2260220328,v1:172.21.15.53:6819/2260220328] exists,up f9403889-7fb8-4c49-8193-55ec6ae285cc', 'osd.4 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/2074654185,v1:172.21.15.53:6801/2074654185] [v2:172.21.15.53:6802/2074654185,v1:172.21.15.53:6803/2074654185] exists,up cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'osd.5 up in weight 1 up_from 12 up_thru 16 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/2633163970,v1:172.21.15.53:6809/2633163970] [v2:172.21.15.53:6810/2633163970,v1:172.21.15.53:6811/2633163970] exists,up ebca2188-456e-4d5c-956d-6382dd41b9bd'] 2024-09-17T21:45:59.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-17T21:45:59.660 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:45:59.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "mon.a", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster": "ceph", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "admin_socket_mode": "", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_client_required": "cephx, none", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_cluster_required": "cephx", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_debug": "true", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_required": "cephx", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-17T21:45:59.661 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_supported": "cephx", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio": "true", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_poll_ms": "250", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_aio_reap_max": "16", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_async_discard": "false", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_block_size": "4096", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio": "false", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_debug_inflight_ios": "false", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_enable_discard": "false", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry": "3", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash": "0", 2024-09-17T21:45:59.662 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_hipri": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bdev_type": "", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_alloc_size": "1048576", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_allocator": "hybrid", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_buffered_io": "true", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_for_zeros": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-17T21:45:59.663 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_compact_log_sync": "false", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_log_runway": "4194304", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_max_prefetch": "1048576", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_flush_size": "524288", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_min_log_runway": "1048576", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery": "false", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluefs_sync_write": "false", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-17T21:45:59.664 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocation_from_file": "true", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_allocator": "hybrid", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi": "true", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_create": "true", 2024-09-17T21:45:59.665 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_create": "false", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_path": "", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_db_size": "0", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_path": "", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_preallocate_file": "false", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_size": "107374182400", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_create": "false", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_path": "", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_block_wal_size": "100663296", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs": "true", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-17T21:45:59.666 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune": "true", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size": "0", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-17T21:45:59.667 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cache_type": "2q", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_clone_cow": "true", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_mode": "none", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_csum_type": "crc32c", 2024-09-17T21:45:59.668 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_freelist": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_small_allocations": "0", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-17T21:45:59.669 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_read": "true", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_default_buffered_write": "false", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fail_eio": "false", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-17T21:45:59.670 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-17T21:45:59.671 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_ignore_data_csum": "false", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_log_op_age": "5.000000", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_alloc_size": "0", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size": "0", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_max_deferred_txc": "32", 2024-09-17T21:45:59.672 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size": "0", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_nid_prealloc": "1024", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_retry_disk_reads": "3", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cf": "true", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-17T21:45:59.673 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_spdk_mem": "512", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_tracing": "false", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-17T21:45:59.674 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "bluestore_zero_block_detection": "false", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-17T21:45:59.675 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_signatures": "false", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_cluster_require_version": "2", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_signatures": "false", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_require_version": "2", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_signatures": "false", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_service_require_version": "2", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "cephx_sign_messages": "true", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "chdir": "", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client_acl_type": "", 2024-09-17T21:45:59.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client_asio_thread_count": "2", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_mid": "0.750000", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_cache_size": "16384", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_caps_release_delay": "5", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_check_pool_perm": "true", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_force_sync_read": "false", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_getattr_caps": "false", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_features": "", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_debug_inject_tick_delay": "0", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_die_on_failed_remount": "false", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_dirsize_rbytes": "true", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_force_lazyio": "false", 2024-09-17T21:45:59.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_fs": "", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_inject_release_failure": "false", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_inline_size": "4096", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mds_namespace": "", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_metadata": "", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_gid": "-1", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_timeout": "300", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mount_uid": "-1", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_mountpoint": "/", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_notify_timeout": "10", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc": "true", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty": "104857600", 2024-09-17T21:45:59.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_max_objects": "1000", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_size": "209715200", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_oc_target_dirty": "8388608", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_permissions": "true", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota": "true", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_quota_df": "true", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_bytes": "0", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_max_periods": "4", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_readahead_min": "131072", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_reconnect_stale": "false", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_shutdown_timeout": "30", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_snapdir": ".snap", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_tick_interval": "1", 2024-09-17T21:45:59.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_trace": "", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_try_dentry_invalidate": "false", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_faked_inos": "false", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_use_random_mds": "false", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog": "false", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_graylog_port": "12201", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_monitors": "default=true", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog": "false", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "clog_to_syslog_level": "info", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "-", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network": "", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_network_interface": "", 2024-09-17T21:45:59.680 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_isal": "false", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_level": "5", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zlib_winsize": "-15", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "compressor_zstd_level": "1", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_op_num_threads": "6", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_osd_stat_interval": "0", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_cpu_cores": "", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crimson_seastar_num_threads": "0", 2024-09-17T21:45:59.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook": "", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location_hook_timeout": "10", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "daemonize": "false", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "daos_pool": "tank", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_alienstore": "0/5", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_allow_any_pool_priority": "false", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok": "1/5", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asok_assert_abort": "false", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_asserts_on_shutdown": "true", 2024-09-17T21:45:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_auth": "20/20", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bdev": "1/3", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluefs": "1/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_bluestore": "1/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_buffer": "0/1", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ceph_exporter": "1/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephfs_mirror": "0/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cephsqlite": "0/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_client": "0/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_compressor": "1/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_context": "0/1", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crimson_interrupt": "0/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crush": "1/1", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_crypto": "1/5", 2024-09-17T21:45:59.683 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_cyanstore": "0/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_deliberately_leak_memory": "false", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_disable_randomized_ping": "false", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_dpdk": "1/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_eventtrace": "1/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filer": "0/1", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_filestore": "1/3", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_finisher": "1/1", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_fuse": "1/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeat_testing_span": "0", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_heartbeatmap": "1/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_javaclient": "1/5", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journal": "1/3", 2024-09-17T21:45:59.684 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_journaler": "0/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_kstore": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_lockdep": "0/1", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mclock": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_balancer": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_locker": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_log_expire": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_migrator": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mds_quiesce": "3/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_memstore": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgr": "2/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mgrc": "1/5", 2024-09-17T21:45:59.685 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_mon": "20/20", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_monc": "0/10", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_ms": "1/1", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_none": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objclass": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objectcacher": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_objecter": "0/1", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_optracker": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_osd": "1/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_paxos": "20/20", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_perfcounter": "1/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_prioritycache": "1/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rados": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd": "0/5", 2024-09-17T21:45:59.686 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_mirror": "0/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_pwl": "0/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rbd_replay": "0/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_refs": "0/0", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_reserver": "1/1", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_access": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_datacache": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_dbstore": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_flight": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rgw_sync": "1/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_rocksdb": "4/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore": "0/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_backref": "0/5", 2024-09-17T21:45:59.687 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cache": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_cleaner": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_device": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_epm": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_journal": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_lba": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_odata": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_omap": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_onode": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_t": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_seastore_tm": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_striper": "0/1", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_test": "0/5", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_throttle": "1/1", 2024-09-17T21:45:59.688 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_timer": "0/1", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_tp": "0/5", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "debug_trace": "1/5", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "device_failure_prediction_mode": "none", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_graylog": "false", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_journald": "false", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_stderr": "true", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "err_to_syslog": "false", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event_tracing": "false", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_addr": "0.0.0.0", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_http_port": "9926", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_prio_limit": "5", 2024-09-17T21:45:59.689 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_sort_metrics": "true", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "exporter_stats_period": "5", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "fake_statfs_for_testing": "0", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "fatal_signal_handlers": "true", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_purge_ops": "10", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filer_max_truncate_ops": "128", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_apply_finisher_threads": "1", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_blackhole": "false", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_clone_range": "true", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_btrfs_snap": "true", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_caller_concurrency": "10", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_collect_device_partition_information": "true", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_commit_timeout": "600.000000", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_inject_read_err": "false", 2024-09-17T21:45:59.690 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_omap_check": "false", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_debug_verify_split": "false", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_dump_file": "", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fadvise": "true", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fail_eio": "true", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_shards": "16", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fd_cache_size": "128", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap": "false", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fiemap_threshold": "4096", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_inject_stall": "0", 2024-09-17T21:45:59.691 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_parallel": "false", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_trailing": "false", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_journal_writeahead": "false", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_kill_at": "0", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs": "0", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size": "0", 2024-09-17T21:45:59.692 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_merge_threshold": "-10", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_odsync_write": "false", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend": "rocksdb", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_backend_path": "", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_thread_timeout": "60", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_op_threads": "2", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_punch_hole": "false", 2024-09-17T21:45:59.693 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_queue_max_ops": "50", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_seek_data_hole": "false", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc": "false", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-17T21:45:59.694 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_splice": "false", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_multiple": "2", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_split_rand_factor": "20", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_update_to": "1000", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_enable": "true", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-17T21:45:59.695 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_xfs_extsize": "false", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "filestore_zfs_snap": "false", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fio_dir": "/tmp/fio", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_allow_other": "true", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_atomic_o_trunc": "true", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_big_writes": "true", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_debug": "false", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_default_permissions": "false", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_disable_pagecache": "false", 2024-09-17T21:45:59.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_max_write": "0", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_multithreaded": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_require_active_mds": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_set_user_groups": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_move": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_read": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_splice_write": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "fuse_use_invalidate_cb": "true", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "gss_target_name": "ceph", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_file": "", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_inject_failure": "0", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_interval": "5", 2024-09-17T21:45:59.697 INFO:teuthology.orchestra.run.smithi007.stdout: "host": "smithi007", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "inject_early_sigterm": "false", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_agent_port": "6799", 2024-09-17T21:45:59.698 INFO:teuthology.orchestra.run.smithi007.stdout: "jaeger_tracing_enable": "false", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_aio": "true", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_align_min_size": "65536", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_align": "true", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_block_size": "4096", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_dio": "true", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_discard": "false", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_force_aio": "false", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_ignore_corruption": "false", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_bytes": "10485760", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_max_write_entries": "100", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_replay_from": "0", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-17T21:45:59.699 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_write_header_frequency": "0", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "journal_zero_on_create": "false", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prefetch_periods": "10", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_prezero_periods": "5", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "journaler_write_head_interval": "15", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "keyfile": "", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_backend": "rocksdb", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_default_stripe_size": "65536", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount": "false", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_bytes": "67108864", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_max_ops": "512", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_nid_prealloc": "1024", 2024-09-17T21:45:59.700 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_onode_map_size": "1024", 2024-09-17T21:45:59.701 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-17T21:45:59.701 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_submit_transaction": "false", 2024-09-17T21:45:59.701 INFO:teuthology.orchestra.run.smithi007.stdout: "kstore_sync_transaction": "false", 2024-09-17T21:45:59.701 INFO:teuthology.orchestra.run.smithi007.stdout: "librados_thread_count": "2", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep": "false", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "lockdep_force_backtrace": "false", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_coarse_timestamps": "true", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_flush_on_exit": "false", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_host": "127.0.0.1", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_graylog_port": "12201", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_new": "1000", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_max_recent": "10000", 2024-09-17T21:45:59.704 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stderr_prefix": "", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_stop_at_utilization": "0.970000", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_file": "true", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_graylog": "false", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_journald": "false", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_stderr": "false", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "log_to_syslog": "false", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "max_rotating_auth_attempts": "10", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_action_on_write_error": "1", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_alternate_name_max": "8192", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_asio_thread_count": "2", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_export_pin": "true", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_dirs": "true", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-17T21:45:59.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_interval": "5", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_interval": "10", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max": "-1", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_max_until": "-1", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_merge_size": "50", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_midchunk": "0.300000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_min_start": "0.200000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_minchunk": "0.001000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_mode": "0", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_max": "1.200000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_need_min": "0.800000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_overload_epochs": "2", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-17T21:45:59.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_bits": "3", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_size": "10000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_target_decay": "10.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_grace": "15.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_interval": "4.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_mid": "0.700000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_delay": "0", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-17T21:45:59.707 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_release_free_interval": "10", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_reservation": "0.050000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_interval": "1", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cache_trim_threshold": "262144", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_prealloc_inos": "1000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_connect_bootstrapping": "false", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_damage_table_max_entries": "10000", 2024-09-17T21:45:59.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_auth_pins": "false", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_frag": "false", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_scatterstat": "false", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_debug_subtrees": "false", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_decay_halflife": "5.000000", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_default_dir_hash": "2", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_defer_session_stale": "true", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_deny_all_reconnect": "false", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_keys_per_op": "16384", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_commit_size": "10", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_max_entries": "0", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dir_prefetch": "true", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-17T21:45:59.709 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_on_map": "false", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_early_reply": "true", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enable_op_tracker": "true", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_enforce_unique_name": "true", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random": "true", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-17T21:45:59.710 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_health_summarize_threshold": "10", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_health_dummy": "false", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_migrator_session_race": "false", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_join_fs": "", 2024-09-17T21:45:59.711 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_journal_format": "1", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_create_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_export_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_import_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_expire_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_journal_replay_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_link_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_mdstable_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_openc_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_rename_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_kill_shutdown_at": "0", 2024-09-17T21:45:59.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_event_large_threshold": "524288", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_events_per_segment": "1024", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_events": "-1", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_max_segments": "128", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_pause": "false", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_segment_size": "0", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_threshold": "128", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_log_warn_factor": "2.000000", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_caps_per_client": "1048576", 2024-09-17T21:45:59.713 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_flushes": "100000", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_completed_requests": "100000", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_export_size": "20971520", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_file_recover": "32", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_files": "64", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops": "8192", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_max_snaps_per_dir": "100", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_metrics_update_interval": "2", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_per_client": "100", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_min_caps_working_set": "10000", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_numa_node": "-1", 2024-09-17T21:45:59.714 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_complaint_time": "30.000000", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_duration": "600", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_size": "20", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_size": "20", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_op_log_threshold": "5", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_grace": "15", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_ping_interval": "5", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_caps": "30000", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-17T21:45:59.715 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_recall_warning_threshold": "262144", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_interval": "1.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_gid": "0", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_root_ino_uid": "0", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-17T21:45:59.716 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_shutdown_check": "0", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_skip_ino": "0", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_max_uid": "4294967294", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_min_uid": "0", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_snap_rstat": "false", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_standby_replay_damaged": "false", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_symlink_recovery": "true", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_exports": "0", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_thrash_fragments": "0", 2024-09-17T21:45:59.717 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_tick_interval": "5.000000", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_valgrind_exit": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_backtrace": "1", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_verify_scatter": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mds_wipe_sessions": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mempool_debug": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_device_bytes": "1073741824", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_set": "false", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "memstore_page_size": "65536", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_bytes": "134217728", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_messages": "512", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-17T21:45:59.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_disabled_modules": "", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_enable_op_tracker": "true", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_max_pg_num_change": "128", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_bytes": "134217728", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mds_messages": "128", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_bytes": "134217728", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_mon_messages": "128", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_duration": "600", 2024-09-17T21:45:59.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_size": "20", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_op_log_threshold": "5", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_bytes": "536870912", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_osd_messages": "8192", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_pool": "true", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_standby_modules": "true", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_period": "5", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_stats_threshold": "5", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_tick_period": "2", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-17T21:45:59.720 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_delete": "true", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_allow_pool_size_one": "true", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_auth_validate_all_caps": "true", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_bytes": "104857600", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_directed_command_retry": "5", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_hunt_parallel": "3", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_log_interval": "1.000000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-17T21:45:59.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_interval": "10.000000", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_client_target_rank": "-1", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_file_level": "debug", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_file": "true", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_journald": "false", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-17T21:45:59.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_bootstrap": "false", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_start": "false", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_compact_on_trim": "true", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_cpu_threads": "4", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_crush_min_required_version": "hammer", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_daemon_bytes": "419430400", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_crit": "5", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_avail_warn": "5", 2024-09-17T21:45:59.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_data_size_warn": "16106127360", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_json": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_dump_transactions": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_extra_checks": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_reef": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_no_require_squid": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-17T21:45:59.724 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_mkfs_grace": "120", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_down_uptime_grace": "60", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_default_strategy": "1", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_election_timeout": "5.000000", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_divisor": "2", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_enable_op_tracker": "true", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fake_pool_delete": "false", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_force_quorum_join": "false", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_globalid_prealloc": "10000", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_detail_to_clog": "false", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_log_update_period": "5", 2024-09-17T21:45:59.725 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_max_detail": "50", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog": "true", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_interval": "600", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host": "172.21.15.7,[v2:172.21.15.7:3301,v1:172.21.15.7:6790],172.21.15.53", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_host_override": "", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_initial_members": "", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease": "5.000000", 2024-09-17T21:45:59.726 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_full_interval": "50", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max": "10000", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_log_max_summary": "50", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_log_epochs": "500", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_osd": "10000", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pg_per_osd": "10000", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_pool_pg_num": "65536", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-17T21:45:59.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_force_trim_to": "0", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mds_skip_sanity": "false", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_autotune": "true", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_memory_target": "2147483648", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_beacon_grace": "30", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_digest_period": "5", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_inactive_grace": "60", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_min_osdmap_epochs": "500", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_complaint_time": "30", 2024-09-17T21:45:59.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_duration": "600", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_size": "20", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_size": "20", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_op_log_threshold": "5", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_in": "false", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size": "500", 2024-09-17T21:45:59.729 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_interval": "600", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_force_trim_to": "0", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-17T21:45:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_down_reporters": "2", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_report_timeout": "900", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-17T21:45:59.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_stuck_threshold": "60", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_probe_timeout": "2.000000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_change": "0.050000", 2024-09-17T21:45:59.732 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_max_osds": "4", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_interval": "86400", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_max_keys": "100", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_scrub_timeout": "300", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_session_timeout": "300", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_smart_report_timeout": "5", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stat_smooth_intervals": "6", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-17T21:45:59.733 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_min_size": "2", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_pool_size": "4", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_debug": "false", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_provider_kill_at": "0", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_requester_kill_at": "0", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_sync_timeout": "60.000000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_target_pg_per_osd": "100", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_tick_interval": "5", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_interval": "300.000000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-17T21:45:59.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_use_min_delay_socket": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_older_version_delay": "604800", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_misplaced": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_older_version": "true", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-17T21:45:59.735 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": "", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-17T21:45:59.736 INFO:teuthology.orchestra.run.smithi007.stdout: "motr_tracing_enabled": "false", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_op_threads": "3", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_cm": "false", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_device_name": "", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_dscp": "96", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_local_gid": "", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_port_num": "1", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-17T21:45:59.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_sl": "3", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_support_srq": "true", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_rdma_type": "ib", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_async_reap_threshold": "5", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_before_connect": "false", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv4": "true", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_ipv6": "false", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr1": "true", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_msgr2": "true", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_max": "7568", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_port_min": "6800", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_count": "3", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_bind_retry_delay": "5", 2024-09-17T21:45:59.738 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_client": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mds": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mgr": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_mon": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_blackhole_osd": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_client_mode": "crc secure", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_mode": "crc secure", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_cluster_type": "", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_compress_secure": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_idle_timeout": "900", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_connection_ready_timeout": "10", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_data": "true", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_crc_header": "true", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bad_msg": "false", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_bug": "true", 2024-09-17T21:45:59.739 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_old_message": "true", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_skipped_message": "false", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_coremask": "0xF", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_enable_tso": "true", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hugepages": "", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_lro": "true", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_memory_channel": "4", 2024-09-17T21:45:59.740 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_pmd": "", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_port_id": "0", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_dump_on_send": "false", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_initial_backoff": "0.200000", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_max": "1.000000", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_delay_type": "", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_network_congestion": "0", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_inject_socket_failures": "5000", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_learn_addr_from_peer": "true", 2024-09-17T21:45:59.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_accept_failures": "4", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_max_backoff": "15.000000", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_client_mode": "secure crc", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_mon_service_mode": "secure crc", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_min_size": "1024", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compress_mode": "none", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_pq_min_cost": "65536", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_public_type": "", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_service_mode": "crc secure", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_listen_backlog": "512", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_nodelay": "true", 2024-09-17T21:45:59.742 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_tcp_rcvbuf": "0", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "ms_type": "async+posix", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "no_config_file": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_completion_locks_per_session": "32", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inflight_ops": "1024", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_tick_interval": "5.000000", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objecter_timeout": "10.000000", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_blackhole": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-17T21:45:59.743 INFO:teuthology.orchestra.run.smithi007.stdout: "openssl_engine_opts": "", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_delay_time": "5.000000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_hist_halflife": "1000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_low_ops": "2", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_max_ops": "4", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_agent_slop": "0.020000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_asio_thread_count": "2", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_async_recovery_min_cost": "100", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_max": "512", 2024-09-17T21:45:59.744 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backfill_scan_min": "64", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_degraded": "false", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_peering": "false", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_backoff_on_unfound": "true", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_beacon_report_interval": "300", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_duration": "30", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_max_block_size": "67108864", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blkin_trace_all": "false", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_for_log_corruption": "false", 2024-09-17T21:45:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_class_update_on_start": "true", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_cap": "256", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_message_size_cap": "524288000", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_op_priority": "63", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_client_watch_timeout": "30", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_command_max_records": "256", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_compact_on_start": "false", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-17T21:45:59.746 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_on_start": "true", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_crush_update_weight_set": "true", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_feed_pullee": "-1", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_misdirected_ops": "false", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_acting_change": "false", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_no_purge_strays": "false", 2024-09-17T21:45:59.747 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_op_order": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_shutdown": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_trim_objects": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_snaps": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_keys": "1024", 2024-09-17T21:45:59.748 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_stride": "524288", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_default_notify_timeout": "30", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep": "0.000000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_discard_disconnected_ops": "true", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_enable_op_tracker": "true", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-17T21:45:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_info": "true", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown": "true", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_function_tracing": "false", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_grace": "20", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_interval": "6", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_peers": "10", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_min_size": "2000", 2024-09-17T21:45:59.750 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_stale": "600", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_max_size": "100000", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_min_size": "1000", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_journal_size": "5120", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_kill_backfill_at": "0", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_cache_size": "50", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_dedup": "true", 2024-09-17T21:45:59.751 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max": "40", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_map_share_max_epochs": "40", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_name_len": "100", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_attr_size": "0", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_backfills": "1", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_count": "5", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_markdown_period": "600", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_name_len": "2048", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_namespace_len": "256", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_object_size": "134217728", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-17T21:45:59.752 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_blocked_by": "16", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_log_entries": "10000", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_pgls": "1024", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_cost": "8388608", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_push_objects": "10", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_scrubs": "3", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_trimming_pgs": "2", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_op_reply_len": "64", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_max_write_size": "90", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-17T21:45:59.753 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_profile": "balanced", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-17T21:45:59.754 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_base": "805306368", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_min": "134217728", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target": "4294967296", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_autotune": "false", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_pg_log_entries": "250", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_min_recovery_priority": "0", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_interval": "5", 2024-09-17T21:45:59.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_cache_shards": "32", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_num_op_tracker_shard": "32", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_auto_affinity": "true", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_node": "-1", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_numa_prefer_iface": "true", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objecter_finishers": "1", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore": "bluestore", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_fuse": "false", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_objectstore_tracing": "false", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_complaint_time": "30.000000", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_duration": "600", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_size": "20", 2024-09-17T21:45:59.756 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_size": "20", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_log_threshold": "5", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards": "0", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_hdd": "5", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_shards_ssd": "8", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_pq_min_cost": "65536", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_queue_cut_off": "high", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_op_thread_timeout": "15", 2024-09-17T21:45:59.757 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_open_classes_on_start": "true", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_os_flags": "0", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_peering_op_priority": "255", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_cost": "1048576", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_delete_priority": "5", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_max": "10000", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_log_trim_min": "100", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-17T21:45:59.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crimson": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_flags": "0", 2024-09-17T21:45:59.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_min_size": "0", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pg_num": "32", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_pgp_num": "0", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_read_ratio": "70", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_size": "2", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_default_type": "replicated", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_push_per_object_cost": "1000", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap": "true", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-17T21:45:59.760 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_cost": "20971520", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active": "0", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_max_single_start": "1", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_priority": "3", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_priority": "5", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep": "0.000000", 2024-09-17T21:45:59.761 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_repair_during_recovery": "false", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_requested_scrub_priority": "120", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair": "false", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_hour": "0", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_begin_week_day": "0", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_max": "25", 2024-09-17T21:45:59.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_chunk_min": "5", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_cost": "52428800", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_during_recovery": "false", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_hour": "0", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_end_week_day": "0", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_event_cost": "4096", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_invalid_stats": "true", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_max_preemptions": "5", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_priority": "5", 2024-09-17T21:45:59.763 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_scrub_sleep": "0.000000", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_skip_data_digest": "false", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_smart_report_timeout": "5", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_cost": "1048576", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_priority": "5", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-17T21:45:59.764 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_target_transaction_size": "30", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_tracing": "false", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_use_stale_snap": "false", 2024-09-17T21:45:59.765 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "osdc_blkin_trace_all": "false", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_kill_at": "0", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_max_join_drift": "10", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min": "500", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_min_wait": "0.050000", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_propose_interval": "1.000000", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max": "500", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_service_trim_min": "250", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_stash_full_interval": "25", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_max": "500", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "paxos_trim_min": "250", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "perf": "true", 2024-09-17T21:45:59.766 INFO:teuthology.orchestra.run.smithi007.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "-", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrv": "", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_bind_addr": "-", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network": "", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_network_interface": "", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_busy_polling": "false", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_enabled": "false", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "qat_compressor_session_max_number": "256", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_mon_op_timeout": "0", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_osd_op_timeout": "0", 2024-09-17T21:45:59.767 INFO:teuthology.orchestra.run.smithi007.stdout: "rados_tracing": "false", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_atime_update_interval": "60", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_parent_reads": "false", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_balance_snap_reads": "false", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blkin_trace_all": "false", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache": "true", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_policy": "writearound", 2024-09-17T21:45:59.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_size": "33554432", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_clone_copy_on_read": "false", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_compression_hint": "none", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_concurrent_management_ops": "10", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_clone_format": "auto", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_data_pool": "", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_features": "61", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_format": "2", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_map_options": "", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_order": "22", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_pool": "rbd", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-17T21:45:59.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_count": "0", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_default_stripe_unit": "0", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_enable_alloc_hint": "true", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler": "simple", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-17T21:45:59.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_order": "24", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_pool": "", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_journal_splay_width": "4", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_parent_reads": "false", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_localize_snap_reads": "false", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-17T21:45:59.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-17T21:45:59.772 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_mtime_update_interval": "60", 2024-09-17T21:45:59.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_non_blocking_aio": "true", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_thread_timeout": "60", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_op_threads": "1", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_parent_cache_enabled": "false", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_plugins": "", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst": "0", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_bps_limit": "0", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_exclude_ops": "0", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst": "0", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-17T21:45:59.774 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_iops_limit": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-17T21:45:59.775 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_read_from_replica_policy": "default", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_skip_partial_discard": "true", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_tracing": "false", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_names": "true", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd_validate_pool": "true", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_base_url": "", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "restapi_log_level": "", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-17T21:45:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_acl_grants_max_num": "100", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_admin_entry": "admin", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_backend_store": "rados", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_barbican_url": "", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_beast_enable_async": "true", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache": "false", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-17T21:45:59.777 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_enabled": "true", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_expiry_interval": "900", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cache_lru_size": "25000", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_config_store": "rados", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_content_length_compat": "false", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress": "true", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cors_rules_max_num": "100", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_cross_domain_policy": "", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_addr": "", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-17T21:45:59.778 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_password": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_kmip_username": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_require_ssl": "true", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-17T21:45:59.779 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_addr": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_auth": "token", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_namespace": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_prefix": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-17T21:45:59.780 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_token_file": "", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_buffersize": "524288", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_low_speed_time": "300", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-17T21:45:59.781 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_d4n_port": "6379", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_changes_size": "1000", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_num_shards": "128", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_log_window": "30", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-17T21:45:59.782 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-17T21:45:59.783 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_name": "", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dns_s3website_name": "", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_dynamic_resharding": "true", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_gc_threads": "true", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_lc_threads": "true", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_ops_log": "false", 2024-09-17T21:45:59.784 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_quota_threads": "true", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_static_website": "false", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enable_usage_log": "false", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_enforce_swift_acls": "true", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_exit_timeout_secs": "120", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_expose_bucket": "false", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_extended_http_attrs": "", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_filter": "none", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_frontends": "beast port=7480", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred": "50", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_objs": "32", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-17T21:45:59.785 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_gc_processor_period": "3600", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_init_timeout": "300", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_connection_idle": "300", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-17T21:45:59.786 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_domain": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_password_path": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_project": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_tenant": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_token_path": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_admin_user": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_api_version": "2", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_domain": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_password": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_project": "", 2024-09-17T21:45:59.787 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_barbican_user": "", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_url": "", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_debug_interval": "-1", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_lock_max_time": "90", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_objs": "32", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_rules": "1000", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_worker": "3", 2024-09-17T21:45:59.788 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lc_thread_delay": "0", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_dnattr": "uid", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_searchfilter": "", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ldap_uri": "", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_http_headers": "", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-17T21:45:59.789 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_log_object_name_utc": "false", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_name_len": "0", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attr_size": "0", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_chunk_size": "4194304", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_listing_results": "1000", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_notify_retries": "10", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_param_size": "1048576", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_put_size": "5368709120", 2024-09-17T21:45:59.790 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_max_slo_entries": "1000", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_log_max_shards": "64", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_mp_lock_max_time": "600", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-17T21:45:59.791 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_max_gc": "300", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_async_rados_threads": "32", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_num_control_oids": "8", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_numa_node": "-1", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-17T21:45:59.792 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_chunk_size": "100", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_gc_interval": "600", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_thread_timeout": "600", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_op_tracing": "false", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_token": "", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_url": "", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_opa_verify_ssl": "true", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_rados": "false", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_ops_log_socket_path": "", 2024-09-17T21:45:59.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval": "2.000000", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lanes": "3", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_cache_partitions": "3", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_continue": "true", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-17T21:45:59.794 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rados_tracing": "false", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm": "", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_id": "", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region": "", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_request_uri": "", 2024-09-17T21:45:59.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_batch_size": "64", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_max_aio": "128", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_num_logs": "16", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_reshard_thread_interval": "600", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_resolve_cname": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_run_sync_thread": "true", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-17T21:45:59.796 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_scheduler_type": "throttler", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_script_uri": "", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_service_provider_name": "", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_id": "", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_client_secret": "", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_entry": "sts", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_key": "sts", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_min_session_duration": "900", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sts_token_introspection_url": "", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_account_in_url": "false", 2024-09-17T21:45:59.797 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_entry": "auth", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_auth_url": "", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_custom_header": "", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_need_stats": "true", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_tenant_name": "", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_token_expiration": "86400", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url": "", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_url_prefix": "swift", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_lease_period": "120", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-17T21:45:59.798 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_thread_pool_size": "512", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_comment": "", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_createby": "", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_encoding": "", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_flag": "false", 2024-09-17T21:45:59.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_origin": "", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_torrent_tracker": "", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_trust_forwarded_https": "false", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_shards": "32", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_usage_max_user_shards": "1", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_use_opa_authz": "false", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache": "false", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_max_buckets": "1000", 2024-09-17T21:45:59.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_policies_max_num": "100", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_user_unique_email": "true", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_verify_ssl": "true", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone": "", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_id": "", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup": "", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_id": "", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-17T21:45:59.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_auto_session_clear": "true", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_block_size": "4096", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_size": "536870912", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-17T21:45:59.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_index_type": "binary_search", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_partition_filters": "false", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_perf": "false", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "run_dir": "/var/run/ceph", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_block_create": "true", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cache_lru_size": "67108864", 2024-09-17T21:45:59.803 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_max_object_size": "16777216", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_device_size": "53687091200", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_full_integrity_check": "false", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_capacity": "16", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_main_device_type": "SSD", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-17T21:45:59.804 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "seastore_segment_size": "67108864", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "set_keepcaps": "false", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "setgroup": "", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser": "", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "setuser_match_path": "", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "thp": "false", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_default_timeout": "60", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout: "throttler_perf_counter": "true" 2024-09-17T21:45:59.805 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:45:59.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-17T21:45:59.923 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-17T21:45:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:45:59.933 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-17T21:45:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:45:59.941 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-17T21:45:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:45:59.952 DEBUG:teuthology.orchestra.run.smithi007:> type ceph-objectstore-tool 2024-09-17T21:45:59.967 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:45:59.967 DEBUG:teuthology.orchestra.run.smithi053:> type ceph-objectstore-tool 2024-09-17T21:45:59.985 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:45:59.985 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-17T21:45:59.995 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-17T21:45:59.995 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-17T21:45:59.995 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:45:59.995 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:45:59.996 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:45:59.996 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 1.000000 2024-09-17T21:45:59.996 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T21:45:59.996 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-17T21:45:59.996 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-17T21:45:59.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:45:59.997 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-17T21:45:59.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:45:59.997 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-17T21:45:59.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:45:59.998 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 512 -w 512 -t 512 -h 512 -l 250000000 -S 0 -N 6000 -M pool_client.0 image_client.0 2024-09-17T21:46:00.027 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:00.024+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:00.063 INFO:teuthology.orchestra.run.smithi165.stdout:mapped writes DISABLED 2024-09-17T21:46:00.064 INFO:teuthology.orchestra.run.smithi165.stdout:mapped reads DISABLED 2024-09-17T21:46:00.064 INFO:teuthology.orchestra.run.smithi165.stdout:Seed set to 4644 2024-09-17T21:46:00.064 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-nbd mode enabled 2024-09-17T21:46:00.098 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:00.095+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:00.123 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:00.123 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:00.123 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:00.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:00.161 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:00.161 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:00.177 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:00.177 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:00.198 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:00.195+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:00.199 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:00.199 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:00.234 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:00.234 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:00.252 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:00.252 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:00.276 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:00.276 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:00.280 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:00.280 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:00.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:00.300 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:00.297+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:00.401 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:00.398+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:00.439 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:00.439 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:00.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:00.502 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:00.499+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:00.555 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:00.555 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:00.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:00.602 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:00.600+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:00.703 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:00.700+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:00.711 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:00.711 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:00.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:00.804 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:00.801+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:00.868 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:00.868 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:00.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:00.905 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:00.902+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:01.006 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:01.003+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:01.025 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.025 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:01.107 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:01.105+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:01.181 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.181 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:01.208 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:01.206+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:01.261 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T21:46:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:46:01.309 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:01.306+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:01.310 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.310 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:01.410 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:01.407+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:01.440 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.440 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:01.511 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:01.508+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:01.557 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.557 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:01.612 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:01.609+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:01.672 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.672 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:01.724 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:01.722+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:01.788 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.789 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:01.838 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:01.835+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:01.914 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:01.911+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:01.944 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:01.944 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:01.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:02.015 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:02.012+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:02.064 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:02.064 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:02.072 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:02.116 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:02.113+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:02.179 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:02.180 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:02.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:02.217 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:02.214+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:02.293 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:02.295 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:02.296 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:02.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:02.338 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:02.335+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:02.412 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:02.412 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:02.418 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:02.416+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:02.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:02.428 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:02.428 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:02.448 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:02.448 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:02.465 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:02.465 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:02.482 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:02.482 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:02.498 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:02.499 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:02.516 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:02.516 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:02.518 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:02.515+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:02.531 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:02.531 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:02.531 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:02.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:02.620 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:02.617+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:02.688 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:02.689 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:02.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:02.721 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:02.718+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-17T21:46:02.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.529367224, 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001138242, 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273178+0000", 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1299999999999999e-07 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273284+0000", 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106878 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273298+0000", 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3602e-05 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273304+0000", 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6500000000000001e-06 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273356+0000", 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2391e-05 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273378+0000", 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1729000000000002e-05 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.274268+0000", 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089036499999999995 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.274315+0000", 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6814000000000003e-05 2024-09-17T21:46:02.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:31 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'17, mlcod=14'14)", 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.520617386, 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001134208, 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:02.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5000000000000002e-07 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281935+0000", 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2749999999999998e-06 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281949+0000", 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4667999999999999e-05 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281963+0000", 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4069e-05 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282001+0000", 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7839000000000001e-05 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282016+0000", 2024-09-17T21:46:02.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4304000000000001e-05 2024-09-17T21:46:02.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283000+0000", 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098423299999999994 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283061+0000", 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1069999999999996e-05 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:32 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'18, mlcod=14'14)", 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.520116976000001, 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001232489, 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:02.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282428+0000", 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6000000000000003e-07 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282501+0000", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3473000000000006e-05 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282513+0000", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1942e-05 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282519+0000", 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:46:02.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282571+0000", 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1884e-05 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282591+0000", 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9576e-05 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283611+0000", 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001020389 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283660+0000", 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8433000000000002e-05 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:33 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'19, mlcod=14'14)", 2024-09-17T21:46:02.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.283217+0000", 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.519326588, 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001011613, 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283217+0000", 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283217+0000", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283218+0000", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3900000000000004e-07 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283224+0000", 2024-09-17T21:46:02.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4909999999999996e-06 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283240+0000", 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6285000000000001e-05 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283250+0000", 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8109999999999997e-06 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283311+0000", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0982e-05 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283335+0000", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4066999999999998e-05 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284190+0000", 2024-09-17T21:46:02.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000855725 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284229+0000", 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8612999999999998e-05 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:34 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'20, mlcod=14'16)", 2024-09-17T21:46:02.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.518652784, 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001166865, 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283892+0000", 2024-09-17T21:46:02.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283897+0000", 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4609999999999999e-06 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283911+0000", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3565e-05 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283917+0000", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283967+0000", 2024-09-17T21:46:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0114999999999997e-05 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283996+0000", 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592e-05 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285023+0000", 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010272250000000001 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285058+0000", 2024-09-17T21:46:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5238999999999999e-05 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:35 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'21, mlcod=14'16)", 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.518120116, 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012645860000000001, 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284425+0000", 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.68e-07 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284504+0000", 2024-09-17T21:46:02.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9231000000000001e-05 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284518+0000", 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4185000000000001e-05 2024-09-17T21:46:02.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284524+0000", 2024-09-17T21:46:02.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-17T21:46:02.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284583+0000", 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9759999999999997e-05 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284607+0000", 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3305000000000001e-05 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285654+0000", 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010475840000000001 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285688+0000", 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4334e-05 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:38 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'24, mlcod=14'22)", 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.287809+0000", 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.514735239, 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099438200000000003, 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287809+0000", 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287809+0000", 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287810+0000", 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7899999999999997e-07 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287861+0000", 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1575e-05 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287874+0000", 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3015e-05 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287881+0000", 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3010000000000003e-06 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287932+0000", 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1032000000000002e-05 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.287952+0000", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0225e-05 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.288754+0000", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080246799999999997 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.288803+0000", 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8887000000000001e-05 2024-09-17T21:46:02.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:54 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'32, mlcod=14'29)", 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.300302+0000", 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.502242253, 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095813199999999999, 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300302+0000", 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300302+0000", 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300302+0000", 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7100000000000002e-07 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300340+0000", 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7635999999999997e-05 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300350+0000", 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6879999999999994e-06 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300358+0000", 2024-09-17T21:46:02.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2009999999999999e-06 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300397+0000", 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8998000000000002e-05 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300411+0000", 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4442e-05 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301199+0000", 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000788009 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301260+0000", 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0587000000000001e-05 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.502168274000001, 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001273038, 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300467+0000", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1119999999999995e-05 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300478+0000", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0896999999999999e-05 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300490+0000", 2024-09-17T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1747e-05 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300641+0000", 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015110300000000001 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300671+0000", 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0215000000000001e-05 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301605+0000", 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093397900000000004 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301649+0000", 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3377999999999997e-05 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:56 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'34, mlcod=14'29)", 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.501784009, 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012141910000000001, 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300761+0000", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300765+0000", 2024-09-17T21:46:02.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6890000000000003e-06 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300777+0000", 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300783+0000", 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4910000000000002e-06 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301285+0000", 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050200500000000003 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301309+0000", 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3451e-05 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301943+0000", 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063448900000000002 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301974+0000", 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0929999999999997e-05 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:57 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'35, mlcod=14'33)", 2024-09-17T21:46:02.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.302651+0000", 2024-09-17T21:46:02.865 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.499893036, 2024-09-17T21:46:02.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010263329999999999, 2024-09-17T21:46:02.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302651+0000", 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302651+0000", 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302652+0000", 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302727+0000", 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5617999999999993e-05 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302738+0000", 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0579e-05 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302763+0000", 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5292000000000001e-05 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302811+0000", 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8003000000000001e-05 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.302824+0000", 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3303e-05 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.303614+0000", 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000789476 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.303677+0000", 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3532000000000001e-05 2024-09-17T21:46:02.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:68 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'42, mlcod=14'39)", 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.310173+0000", 2024-09-17T21:46:02.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.492370680000001, 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010338470000000001, 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310173+0000", 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310173+0000", 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310174+0000", 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310180+0000", 2024-09-17T21:46:02.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8919999999999997e-06 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310191+0000", 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.075e-05 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310196+0000", 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310262+0000", 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6160000000000004e-05 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310276+0000", 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4103e-05 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.311144+0000", 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086755299999999997 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.311207+0000", 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.321e-05 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:69 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'43, mlcod=14'39)", 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.310645+0000", 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.491899171, 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097248400000000002, 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310645+0000", 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310645+0000", 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310645+0000", 2024-09-17T21:46:02.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6899999999999997e-07 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310705+0000", 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9786e-05 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310715+0000", 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7429999999999997e-06 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310721+0000", 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3720000000000001e-06 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310772+0000", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0797999999999997e-05 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.310788+0000", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5628999999999999e-05 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.311587+0000", 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00079973399999999997 2024-09-17T21:46:02.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.311617+0000", 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9853e-05 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:71 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'45, mlcod=14'43)", 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.312598+0000", 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.489946017999999, 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095724500000000001, 2024-09-17T21:46:02.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312598+0000", 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312598+0000", 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312598+0000", 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312635+0000", 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.667e-05 2024-09-17T21:46:02.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312645+0000", 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9280000000000005e-06 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312652+0000", 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7320000000000001e-06 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312702+0000", 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0419e-05 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.312718+0000", 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5611999999999999e-05 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.313500+0000", 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000782046 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.313555+0000", 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5373e-05 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:83 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'53, mlcod=14'50)", 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.320667+0000", 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.481877262999999, 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00097805099999999996, 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320667+0000", 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320667+0000", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320667+0000", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320672+0000", 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999998e-06 2024-09-17T21:46:02.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320682+0000", 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8789999999999998e-06 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320690+0000", 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0320000000000003e-06 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320738+0000", 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8364e-05 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.320754+0000", 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5738000000000001e-05 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.321613+0000", 2024-09-17T21:46:02.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000859196 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.321645+0000", 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1448000000000001e-05 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:87 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'57, mlcod=14'53)", 2024-09-17T21:46:02.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.323171+0000", 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.479372924, 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095473800000000001, 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323171+0000", 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323171+0000", 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323172+0000", 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323227+0000", 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5019000000000003e-05 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323236+0000", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5210000000000008e-06 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323242+0000", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8710000000000002e-06 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323278+0000", 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6168000000000003e-05 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.323292+0000", 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3431000000000001e-05 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.324092+0000", 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080047600000000005 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.324126+0000", 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3730999999999999e-05 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:89 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'59, mlcod=14'57)", 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.325109+0000", 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.477434561000001, 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00099889500000000008, 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325109+0000", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325109+0000", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325110+0000", 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325168+0000", 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7886000000000001e-05 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325178+0000", 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008e-05 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325184+0000", 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2009999999999996e-06 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325225+0000", 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0791999999999998e-05 2024-09-17T21:46:02.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.325241+0000", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5789000000000001e-05 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.326071+0000", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082992000000000005 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.326108+0000", 2024-09-17T21:46:02.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7784999999999998e-05 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:134 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'92, mlcod=14'88)", 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.821106+0000", 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.981437536, 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00096725700000000001, 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821106+0000", 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821106+0000", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821107+0000", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7100000000000002e-07 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821167+0000", 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0408000000000002e-05 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821177+0000", 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3919999999999992e-06 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821182+0000", 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1889999999999998e-06 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821212+0000", 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9909000000000001e-05 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.821224+0000", 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2079000000000001e-05 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822032+0000", 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080765399999999999 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822074+0000", 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2154999999999999e-05 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:137 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'95, mlcod=14'90)", 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.979643547, 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012645969999999999, 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:02.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822901+0000", 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2e-07 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822958+0000", 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6549999999999999e-05 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822969+0000", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1892999999999999e-05 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822974+0000", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1509999999999997e-06 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.823016+0000", 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2521000000000002e-05 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.823036+0000", 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0259e-05 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.824121+0000", 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010846370000000001 2024-09-17T21:46:02.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.824165+0000", 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4066000000000001e-05 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:144 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'102, mlcod=14'98)", 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.828190+0000", 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.974353634, 2024-09-17T21:46:02.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095363800000000003, 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828190+0000", 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828190+0000", 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:02.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828191+0000", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7199999999999999e-07 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828280+0000", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9134000000000004e-05 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828288+0000", 2024-09-17T21:46:02.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4700000000000002e-06 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828292+0000", 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0239999999999999e-06 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828318+0000", 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5304999999999999e-05 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:02.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.828329+0000", 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1253000000000001e-05 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.829101+0000", 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00077199799999999998 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.829144+0000", 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2982000000000003e-05 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:02.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:02.922 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:02.923 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:02.924 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:02.920+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:03.024 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:03.022+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:03.031 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:03.125 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:03.122+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:03.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.147 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:03.147 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:03.147 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:03.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:03.226 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:03.224+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:03.263 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.263 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:03.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:03.263 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:03.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:5 2.2 e18/16 2:5cea7035:::rbd_info:head v 18'1, mlcod=0'0)", 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:53.238918+0000", 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.022677259, 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00085952800000000003, 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238918+0000", 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238918+0000", 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238919+0000", 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4600000000000001e-07 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:03.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238925+0000", 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238942+0000", 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6939000000000002e-05 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.238955+0000", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2748000000000001e-05 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.239009+0000", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.239046+0000", 2024-09-17T21:46:03.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6300999999999997e-05 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.239736+0000", 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069050899999999996 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:53.239778+0000", 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1582999999999998e-05 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:03.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:7 2.2 e19/16 2:5cea7035:::rbd_info:head v 19'2, mlcod=0'0)", 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:54.242601+0000", 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.0189945500000004, 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078281900000000003, 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242601+0000", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242601+0000", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242601+0000", 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.04e-07 2024-09-17T21:46:03.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242606+0000", 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.549e-06 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242616+0000", 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0360000000000001e-05 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:03.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242626+0000", 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6860000000000001e-06 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242685+0000", 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9258000000000001e-05 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.242718+0000", 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2790999999999998e-05 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.243341+0000", 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062350000000000003 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:54.243384+0000", 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2471e-05 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:03.274 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:03.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:46:03.328 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:03.325+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:03.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:03.390 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:03.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:03.429 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:03.426+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:03.526 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.526 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:03.526 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:03.526 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:03.526 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.529 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:03.526+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:03.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:03.631 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:03.628+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:03.683 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:03.683 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:03.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:03.683 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4211.0:1 2.3 e17/16 2:c4c92e5a:::rbd_trash:head v 17'1, mlcod=0'0)", 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:52.237149+0000", 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.444345593, 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012716889999999999, 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237149+0000", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237149+0000", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237149+0000", 2024-09-17T21:46:03.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237160+0000", 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0617e-05 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237170+0000", 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5780000000000005e-06 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237181+0000", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1318000000000001e-05 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237348+0000", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016729200000000001 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.237381+0000", 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2311000000000003e-05 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.238361+0000", 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0009802910000000001 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:52.238421+0000", 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9948000000000003e-05 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:03.688 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:03.692 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:03.734 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:03.731+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:03.813 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:03.813 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:03.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:03.832 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:03.829+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:03.930 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:03.931 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:03.933 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:03.930+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:03.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:04.034 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:04.031+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:04.089 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:04.089 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.926677363, 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:04.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:04.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:04.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:34 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.281121+0000", 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.804358058999998, 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048290219999999997, 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 34 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281121+0000", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281121+0000", 2024-09-17T21:46:04.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281121+0000", 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5199999999999996e-07 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281126+0000", 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.989e-06 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281138+0000", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1405e-05 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281145+0000", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2060000000000001e-06 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.283640+0000", 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024948219999999998 2024-09-17T21:46:04.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.283703+0000", 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3362999999999996e-05 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.283802+0000", 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8663e-05 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.284990+0000", 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001187862 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285855+0000", 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000865083 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285900+0000", 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4815999999999999e-05 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285950+0000", 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0161000000000003e-05 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.281159+0000", 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.804319793000001, 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0048942780000000002, 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 35 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281159+0000", 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281159+0000", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281160+0000", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0399999999999996e-07 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281250+0000", 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9952999999999997e-05 2024-09-17T21:46:04.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281261+0000", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1297999999999999e-05 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281269+0000", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0180000000000001e-06 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.284100+0000", 2024-09-17T21:46:04.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002831325 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.284154+0000", 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4129000000000003e-05 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.284243+0000", 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8885e-05 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285987+0000", 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017434029999999999 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285994+0000", 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7800000000000001e-06 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286026+0000", 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1622999999999999e-05 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286053+0000", 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.726e-05 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.281285+0000", 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.804193740999999, 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049716889999999996, 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 36 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281285+0000", 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281285+0000", 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281286+0000", 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0500000000000003e-07 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281291+0000", 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3600000000000004e-06 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281302+0000", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0307e-05 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281307+0000", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285015+0000", 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037077400000000002 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285097+0000", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1939000000000003e-05 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285181+0000", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4258999999999994e-05 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286059+0000", 2024-09-17T21:46:04.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087768199999999996 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286181+0000", 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012174499999999999 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286211+0000", 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0480999999999998e-05 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286257+0000", 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5651e-05 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.804153748000001, 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051382629999999997, 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 37 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:04.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281326+0000", 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281330+0000", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281341+0000", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0665e-05 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281347+0000", 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285455+0000", 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041082610000000002 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285510+0000", 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4737000000000001e-05 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285592+0000", 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2515000000000005e-05 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286064+0000", 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047146999999999998 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286389+0000", 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032487100000000001 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286422+0000", 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2838e-05 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286463+0000", 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1902e-05 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.122 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.320070+0000", 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.765408562000001, 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045415680000000002, 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 88 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320070+0000", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320070+0000", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320071+0000", 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2800000000000003e-07 2024-09-17T21:46:04.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320073+0000", 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.514e-06 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320077+0000", 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.320085+0000", 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7949999999999996e-06 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.323598+0000", 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035127209999999999 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.323656+0000", 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7880999999999997e-05 2024-09-17T21:46:04.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.323729+0000", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3139000000000006e-05 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.324512+0000", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078302099999999996 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.324524+0000", 2024-09-17T21:46:04.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1915e-05 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.324571+0000", 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6857000000000002e-05 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.324612+0000", 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0920999999999997e-05 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470528+0000", 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614951162000001, 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049275040000000001, 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 112 2024-09-17T21:46:04.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470528+0000", 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470528+0000", 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470528+0000", 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470589+0000", 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1109000000000001e-05 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470600+0000", 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0581000000000001e-05 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470607+0000", 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.669e-06 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.473807+0000", 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032006909999999999 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.473860+0000", 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2575999999999999e-05 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.473938+0000", 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7631000000000003e-05 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474937+0000", 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099939199999999999 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475399+0000", 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046247900000000002 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475430+0000", 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0188999999999998e-05 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475455+0000", 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5599999999999999e-05 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614858937000001, 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055199849999999998, 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 113 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470621+0000", 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7000000000000004e-07 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470626+0000", 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0490000000000003e-06 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470633+0000", 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.413e-06 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470640+0000", 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474317+0000", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036775190000000002 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474366+0000", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8820999999999999e-05 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474443+0000", 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6656999999999995e-05 2024-09-17T21:46:04.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475503+0000", 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010601300000000001 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476084+0000", 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058080899999999995 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476116+0000", 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2240999999999998e-05 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476140+0000", 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4138000000000002e-05 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614828993, 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059845510000000003, 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114 2024-09-17T21:46:04.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1500000000000005e-07 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470654+0000", 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470661+0000", 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470666+0000", 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:46:04.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474985+0000", 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043186580000000004 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475039+0000", 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3964e-05 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475133+0000", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4306000000000001e-05 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476574+0000", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014412190000000001 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476585+0000", 2024-09-17T21:46:04.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0852999999999999e-05 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476612+0000", 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7481e-05 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476634+0000", 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1963999999999998e-05 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614802379, 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064121400000000002, 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 115 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1500000000000005e-07 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470737+0000", 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9543000000000002e-05 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470760+0000", 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470766+0000", 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4699999999999999e-06 2024-09-17T21:46:04.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475535+0000", 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047688649999999997 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475586+0000", 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1138999999999999e-05 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475664+0000", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7914e-05 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476984+0000", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013201210000000001 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477027+0000", 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3154000000000001e-05 2024-09-17T21:46:04.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477059+0000", 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1593000000000001e-05 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477089+0000", 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9876e-05 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614698130000001, 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064079999999999996, 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 116 2024-09-17T21:46:04.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3799999999999997e-07 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470786+0000", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470797+0000", 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0589e-05 2024-09-17T21:46:04.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470802+0000", 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8250000000000001e-06 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476175+0000", 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053730119999999999 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476221+0000", 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5456e-05 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476298+0000", 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6938000000000001e-05 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477120+0000", 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082225899999999999 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477129+0000", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3950000000000007e-06 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477163+0000", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3238000000000003e-05 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477189+0000", 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6077000000000001e-05 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.163 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.614663200999999, 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066701130000000001, 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 117 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3399999999999999e-07 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470822+0000", 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470830+0000", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4910000000000005e-06 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470835+0000", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1540000000000001e-06 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476659+0000", 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058232529999999996 2024-09-17T21:46:04.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476707+0000", 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8656000000000003e-05 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476785+0000", 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8014000000000002e-05 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477231+0000", 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044546899999999999 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477434+0000", 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000203477 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477464+0000", 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0261999999999999e-05 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477486+0000", 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1413e-05 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.267230398999999, 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051643319999999998, 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 134 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818249+0000", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818295+0000", 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6e-05 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818301+0000", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6620000000000002e-06 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818322+0000", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0877999999999999e-05 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.820797+0000", 2024-09-17T21:46:04.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024750240000000001 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.820881+0000", 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4895999999999995e-05 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821021+0000", 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139209 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823308+0000", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002287182 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823322+0000", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4503e-05 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823373+0000", 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0670000000000001e-05 2024-09-17T21:46:04.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823413+0000", 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9798000000000001e-05 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.267150438000002, 2024-09-17T21:46:04.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052327600000000004, 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 135 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818329+0000", 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818332+0000", 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7760000000000002e-06 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818337+0000", 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818342+0000", 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821563+0000", 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032210400000000001 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821617+0000", 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3523000000000002e-05 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821691+0000", 2024-09-17T21:46:04.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3685999999999999e-05 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823453+0000", 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017626619999999999 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823506+0000", 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2828999999999999e-05 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823532+0000", 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5525999999999999e-05 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823561+0000", 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9595e-05 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.267128058000001, 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052620200000000001, 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 136 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818355+0000", 2024-09-17T21:46:04.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.997e-06 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818363+0000", 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9079999999999995e-06 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818366+0000", 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822144+0000", 2024-09-17T21:46:04.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037780029999999998 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822207+0000", 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2237999999999996e-05 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822283+0000", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6884000000000005e-05 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823499+0000", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012151550000000001 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823567+0000", 2024-09-17T21:46:04.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8522999999999994e-05 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823590+0000", 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2340999999999998e-05 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823613+0000", 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3346999999999999e-05 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.267103865999999, 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069329190000000001, 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 137 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-07 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818444+0000", 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8785e-05 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818456+0000", 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1542000000000001e-05 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818460+0000", 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0589999999999996e-06 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822570+0000", 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041107909999999999 2024-09-17T21:46:04.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822653+0000", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2986999999999997e-05 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822792+0000", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013813900000000001 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825200+0000", 2024-09-17T21:46:04.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024085249999999999 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825216+0000", 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6308e-05 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825266+0000", 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9221000000000002e-05 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825308+0000", 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2262999999999998e-05 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.267011279999998, 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069445879999999998, 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 138 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818471+0000", 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7429999999999998e-06 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818477+0000", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0159999999999997e-06 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818480+0000", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.818e-06 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823651+0000", 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005171503 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823703+0000", 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1433e-05 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823778+0000", 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5663999999999999e-05 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825316+0000", 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001537973 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825350+0000", 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3673000000000001e-05 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825389+0000", 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8983000000000003e-05 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825412+0000", 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3402000000000001e-05 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.266995181999999, 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071013059999999999, 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 139 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818488+0000", 2024-09-17T21:46:04.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7400000000000002e-06 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818492+0000", 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818495+0000", 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.909e-06 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824054+0000", 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055592949999999997 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824099+0000", 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4974e-05 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824185+0000", 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825469+0000", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001283246 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825482+0000", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825532+0000", 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9475000000000003e-05 2024-09-17T21:46:04.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825585+0000", 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3074e-05 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.266977143999998, 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00759708, 2024-09-17T21:46:04.213 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 140 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8299999999999998e-07 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818554+0000", 2024-09-17T21:46:04.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2130999999999999e-05 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818559+0000", 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7360000000000001e-06 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818562+0000", 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.932e-06 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824488+0000", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059259680000000002 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824572+0000", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4559000000000001e-05 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824692+0000", 2024-09-17T21:46:04.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011944599999999999 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826018+0000", 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013263439999999999 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826046+0000", 2024-09-17T21:46:04.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7908000000000001e-05 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826076+0000", 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9513999999999999e-05 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826099+0000", 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3258999999999999e-05 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.266910781, 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0080377260000000002, 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 141 2024-09-17T21:46:04.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818569+0000", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818572+0000", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6710000000000001e-06 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818577+0000", 2024-09-17T21:46:04.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000004e-06 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818581+0000", 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6020000000000002e-06 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825609+0000", 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070274500000000002 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825656+0000", 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7697000000000001e-05 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825774+0000", 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117489 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826297+0000", 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052359800000000001 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826505+0000", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020754000000000001 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826558+0000", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3292999999999999e-05 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826606+0000", 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7500999999999998e-05 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.226 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:04.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:04.228 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:04.225+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:04.281 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:46:04.327 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:04.325+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:04.376 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:04.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:04.376 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:04.376 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:04.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:04.428 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:04.425+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:04.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:04.529 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:04.527+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:04.532 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:04.532 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:04.532 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:04.533 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:04.533 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:04.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:04.630 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:04.627+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:04.665 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:04.665 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:04.682 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:04.682 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4132286060000001, 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036959290000000001, 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274238+0000", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.829e-06 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274245+0000", 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1810000000000004e-06 2024-09-17T21:46:04.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274262+0000", 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6239000000000002e-05 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274270+0000", 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5399999999999996e-06 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274341+0000", 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1357000000000001e-05 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274349+0000", 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7100000000000007e-06 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276472+0000", 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021225319999999999 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276559+0000", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7515999999999994e-05 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276665+0000", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105621 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277452+0000", 2024-09-17T21:46:04.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078702500000000001 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277862+0000", 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041006899999999999 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277899+0000", 2024-09-17T21:46:04.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7373000000000003e-05 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277932+0000", 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2937e-05 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4370.0:9 3.1a e23/21 ondisk, result = 0)", 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:04.697 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4096947719999999, 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000168856, 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277771+0000", 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8599999999999997e-07 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277776+0000", 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277787+0000", 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0703000000000001e-05 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277794+0000", 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3719999999999999e-06 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277847+0000", 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2476999999999999e-05 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277855+0000", 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3650000000000004e-06 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277939+0000", 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3912999999999994e-05 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:10 3.1c e24/21 3:3831950c:::rbd_directory:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.405316035, 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011563879999999999, 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:04.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282155+0000", 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4500000000000003e-06 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282165+0000", 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0572e-05 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282179+0000", 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4151999999999999e-05 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282246+0000", 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6095999999999999e-05 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282273+0000", 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7549999999999999e-05 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283249+0000", 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000975671 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283305+0000", 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6499999999999998e-05 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:12 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.4011058949999999, 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061919899999999996, 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286364+0000", 2024-09-17T21:46:04.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5260000000000004e-06 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286380+0000", 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5733999999999999e-05 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286388+0000", 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4950000000000008e-06 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286447+0000", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8798999999999997e-05 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286461+0000", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4576000000000001e-05 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286939+0000", 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047777699999999998 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286978+0000", 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9011999999999999e-05 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:13 3.1 e24/21 3:83138b25:::rbd_object_map.111282762508:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.3995737269999999, 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093130700000000003, 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287892+0000", 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287896+0000", 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287907+0000", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0346000000000001e-05 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287913+0000", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.896e-06 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287980+0000", 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7084999999999999e-05 2024-09-17T21:46:04.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288002+0000", 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2112000000000001e-05 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288776+0000", 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077369400000000001 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288822+0000", 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6940999999999997e-05 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.3901533260000001, 2024-09-17T21:46:04.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000328468, 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 23 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297313+0000", 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.792e-06 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297320+0000", 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8719999999999996e-06 2024-09-17T21:46:04.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297334+0000", 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4151999999999999e-05 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297341+0000", 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6780000000000003e-06 2024-09-17T21:46:04.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297408+0000", 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6873999999999997e-05 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297508+0000", 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9580000000000005e-05 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297640+0000", 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013252000000000001 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:25 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'2, mlcod=0'0)", 2024-09-17T21:46:04.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.388336078, 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061023999999999996, 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299130+0000", 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2900000000000002e-07 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299136+0000", 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8850000000000004e-06 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299148+0000", 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2218999999999999e-05 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299163+0000", 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5167000000000001e-05 2024-09-17T21:46:04.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299214+0000", 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1368e-05 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299230+0000", 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5971e-05 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299701+0000", 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047061 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299739+0000", 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8191000000000001e-05 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4455.0:10 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.2418277049999999, 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032502799999999999, 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4455", 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2139432296", 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10 2024-09-17T21:46:04.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:04.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445638+0000", 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04e-06 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445643+0000", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0409999999999997e-06 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445652+0000", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9099999999999994e-06 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445659+0000", 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6019999999999996e-06 2024-09-17T21:46:04.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445727+0000", 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8429999999999994e-05 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445827+0000", 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010010599999999999 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445962+0000", 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013489900000000001 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4455.0:12 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'3, mlcod=24'1)", 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.240407206, 2024-09-17T21:46:04.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058209400000000001, 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:04.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447059+0000", 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0100000000000001e-06 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447067+0000", 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447080+0000", 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3291e-05 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447086+0000", 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.071e-06 2024-09-17T21:46:04.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447136+0000", 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9484999999999998e-05 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447154+0000", 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7827999999999999e-05 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447607+0000", 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045376599999999998 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447640+0000", 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2416000000000003e-05 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4455.0:13 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.203583013, 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033991400000000003, 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:04.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4455", 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2139432296", 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13 2024-09-17T21:46:04.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483883+0000", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483889+0000", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8660000000000002e-06 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483903+0000", 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4428e-05 2024-09-17T21:46:04.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483909+0000", 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0750000000000003e-06 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483992+0000", 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2095999999999995e-05 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.484087+0000", 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4988999999999995e-05 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.484222+0000", 2024-09-17T21:46:04.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013521 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:04.748 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:04.750 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:04.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:04.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:04.752 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:04.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:04.753 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:04.747+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:04.753 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:04.754 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:04.754 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:04.754 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:04.851 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:04.848+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:04.900 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:04.900 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:04.908 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:04.952 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:04.949+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:05.015 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.015 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.024 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:05.053 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:05.050+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:05.132 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.132 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:05.143 INFO:teuthology.orchestra.run.smithi165.stdout:skipping zero size read 2024-09-17T21:46:05.165 INFO:teuthology.orchestra.run.smithi165.stdout:5 write 0x9188000 thru 0x918e5dc (0x65dd bytes) 2024-09-17T21:46:05.165 INFO:teuthology.orchestra.run.smithi165.stdout:6 write 0x60c5400 thru 0x60d3f90 (0xeb91 bytes) 2024-09-17T21:46:05.182 INFO:teuthology.orchestra.run.smithi165.stdout:7 write 0xa0c1a00 thru 0xa0ca9e5 (0x8fe6 bytes) 2024-09-17T21:46:05.185 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:05.182+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:05.207 INFO:teuthology.orchestra.run.smithi165.stdout:10 write 0xe417e00 thru 0xe41eadb (0x6cdc bytes) 2024-09-17T21:46:05.208 INFO:teuthology.orchestra.run.smithi165.stdout:11 write 0xceeea00 thru 0xcef26fe (0x3cff bytes) 2024-09-17T21:46:05.208 INFO:teuthology.orchestra.run.smithi165.stdout:13 punch from 0xcd3e000 to 0xcd45400, (0x7400 bytes) 2024-09-17T21:46:05.208 INFO:teuthology.orchestra.run.smithi165.stdout:14 write 0xa5e4200 thru 0xa5ea3d5 (0x61d6 bytes) 2024-09-17T21:46:05.208 INFO:teuthology.orchestra.run.smithi165.stdout:15 read 0xb317e00 thru 0xb31feff (0x8100 bytes) 2024-09-17T21:46:05.208 INFO:teuthology.orchestra.run.smithi165.stdout:truncating to largest ever: 0xcc10000 2024-09-17T21:46:05.209 INFO:teuthology.orchestra.run.smithi165.stdout:16 trunc from 0xe41ec00 to 0xcc10000 2024-09-17T21:46:05.238 INFO:teuthology.orchestra.run.smithi165.stdout:17 trunc from 0xcc10000 to 0x954e800 2024-09-17T21:46:05.254 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:05.252+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:05.259 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.259 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:05.270 INFO:teuthology.orchestra.run.smithi165.stdout:18 write 0xe0af600 thru 0xe0b75fd (0x7ffe bytes) 2024-09-17T21:46:05.270 INFO:teuthology.orchestra.run.smithi165.stdout:19 read 0x1acf200 thru 0x1acf711 (0x512 bytes) 2024-09-17T21:46:05.270 INFO:teuthology.orchestra.run.smithi165.stdout:21 read 0x9cc9600 thru 0x9cd40a1 (0xaaa2 bytes) 2024-09-17T21:46:05.271 INFO:teuthology.orchestra.run.smithi165.stdout:22 write 0x27b7200 thru 0x27b8bf5 (0x19f6 bytes) 2024-09-17T21:46:05.271 INFO:teuthology.orchestra.run.smithi165.stdout:23 read 0x3b80e00 thru 0x3b898f4 (0x8af5 bytes) 2024-09-17T21:46:05.279 INFO:teuthology.orchestra.run.smithi165.stdout:24 trunc from 0xe0b7600 to 0x5ed3200 2024-09-17T21:46:05.296 INFO:teuthology.orchestra.run.smithi165.stdout:25 read 0x1ff8000 thru 0x2006e2b (0xee2c bytes) 2024-09-17T21:46:05.302 INFO:teuthology.orchestra.run.smithi165.stdout:29 trunc from 0x5ed3200 to 0x9db0000 2024-09-17T21:46:05.313 INFO:teuthology.orchestra.run.smithi165.stdout:30 read 0x4484400 thru 0x4490f57 (0xcb58 bytes) 2024-09-17T21:46:05.314 INFO:teuthology.orchestra.run.smithi165.stdout:31 punch from 0x7095600 to 0x7098e00, (0x3800 bytes) 2024-09-17T21:46:05.314 INFO:teuthology.orchestra.run.smithi165.stdout:32 write 0x6048e00 thru 0x604c16e (0x336f bytes) 2024-09-17T21:46:05.318 INFO:teuthology.orchestra.run.smithi165.stdout:33 read 0x3422e00 thru 0x342d6e5 (0xa8e6 bytes) 2024-09-17T21:46:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:34 read 0x8d24200 thru 0x8d2b2de (0x70df bytes) 2024-09-17T21:46:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:35 write 0x61b4200 thru 0x61b66ed (0x24ee bytes) 2024-09-17T21:46:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:38 write 0x7bf400 thru 0x7c84cb (0x90cc bytes) 2024-09-17T21:46:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:42 write 0x2a7c00 thru 0x2b7a3b (0xfe3c bytes) 2024-09-17T21:46:05.319 INFO:teuthology.orchestra.run.smithi165.stdout:43 read 0x86d5e00 thru 0x86e2aeb (0xccec bytes) 2024-09-17T21:46:05.330 INFO:teuthology.orchestra.run.smithi165.stdout:44 write 0xaa06600 thru 0xaa0c1b2 (0x5bb3 bytes) 2024-09-17T21:46:05.346 INFO:teuthology.orchestra.run.smithi165.stdout:48 write 0xcda9e00 thru 0xcdb937f (0xf580 bytes) 2024-09-17T21:46:05.350 INFO:teuthology.orchestra.run.smithi165.stdout:49 trunc from 0xcdb9400 to 0x9e7ca00 2024-09-17T21:46:05.355 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:05.352+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:05.365 INFO:teuthology.orchestra.run.smithi165.stdout:52 punch from 0x5b70a00 to 0x5b7e800, (0xde00 bytes) 2024-09-17T21:46:05.366 INFO:teuthology.orchestra.run.smithi165.stdout:53 read 0x8a10c00 thru 0x8a1e05f (0xd460 bytes) 2024-09-17T21:46:05.367 INFO:teuthology.orchestra.run.smithi165.stdout:56 punch from 0x1d2d600 to 0x1d36200, (0x8c00 bytes) 2024-09-17T21:46:05.367 INFO:teuthology.orchestra.run.smithi165.stdout:59 read 0x2484000 thru 0x2491bbb (0xdbbc bytes) 2024-09-17T21:46:05.367 INFO:teuthology.orchestra.run.smithi165.stdout:61 punch from 0x5c5f400 to 0x5c62200, (0x2e00 bytes) 2024-09-17T21:46:05.378 INFO:teuthology.orchestra.run.smithi165.stdout:63 write 0xaf08000 thru 0xaf0d229 (0x522a bytes) 2024-09-17T21:46:05.379 INFO:teuthology.orchestra.run.smithi165.stdout:65 read 0x6aa2e00 thru 0x6aa6008 (0x3209 bytes) 2024-09-17T21:46:05.379 INFO:teuthology.orchestra.run.smithi165.stdout:66 write 0x97c8c00 thru 0x97d07e1 (0x7be2 bytes) 2024-09-17T21:46:05.379 INFO:teuthology.orchestra.run.smithi165.stdout:67 write 0x8078600 thru 0x80810d1 (0x8ad2 bytes) 2024-09-17T21:46:05.379 INFO:teuthology.orchestra.run.smithi165.stdout:69 read 0x48b8400 thru 0x48c0e49 (0x8a4a bytes) 2024-09-17T21:46:05.379 INFO:teuthology.orchestra.run.smithi165.stdout:73 write 0x4975200 thru 0x4982e0d (0xdc0e bytes) 2024-09-17T21:46:05.396 INFO:teuthology.orchestra.run.smithi165.stdout:74 write 0xd547600 thru 0xd556200 (0xec01 bytes) 2024-09-17T21:46:05.413 INFO:teuthology.orchestra.run.smithi165.stdout:75 trunc from 0xd556400 to 0x108200 2024-09-17T21:46:05.434 INFO:teuthology.orchestra.run.smithi165.stdout:76 punch from 0x42400 to 0x4d600, (0xb200 bytes) 2024-09-17T21:46:05.435 INFO:teuthology.orchestra.run.smithi165.stdout:79 read 0x1800 thru 0x2c30 (0x1431 bytes) 2024-09-17T21:46:05.441 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.442 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.442 INFO:teuthology.orchestra.run.smithi165.stdout:80 punch from 0x65800 to 0x67200, (0x1a00 bytes) 2024-09-17T21:46:05.443 INFO:teuthology.orchestra.run.smithi165.stdout:81 trunc from 0x108200 to 0x39c6800 2024-09-17T21:46:05.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:05.466 INFO:teuthology.orchestra.run.smithi165.stdout:83 write 0x6307c00 thru 0x630f8cc (0x7ccd bytes) 2024-09-17T21:46:05.471 INFO:teuthology.orchestra.run.smithi165.stdout:86 punch from 0xe0a800 to 0xe0cc00, (0x2400 bytes) 2024-09-17T21:46:05.472 INFO:teuthology.orchestra.run.smithi165.stdout:89 punch from 0x3a47e00 to 0x3a57c00, (0xfe00 bytes) 2024-09-17T21:46:05.473 INFO:teuthology.orchestra.run.smithi165.stdout:90 trunc from 0x630fa00 to 0x59c3000 2024-09-17T21:46:05.495 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:05.492+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:05.497 INFO:teuthology.orchestra.run.smithi165.stdout:91 write 0x7e2000 thru 0x7e6012 (0x4013 bytes) 2024-09-17T21:46:05.501 INFO:teuthology.orchestra.run.smithi165.stdout:93 trunc from 0x59c3000 to 0x27fde00 2024-09-17T21:46:05.523 INFO:teuthology.orchestra.run.smithi165.stdout:96 write 0x2351200 thru 0x235b973 (0xa774 bytes) 2024-09-17T21:46:05.524 INFO:teuthology.orchestra.run.smithi165.stdout:98 trunc from 0x27fde00 to 0x18fbe00 2024-09-17T21:46:05.556 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:05.554+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:05.561 INFO:teuthology.orchestra.run.smithi165.stdout:99 write 0x4e04e00 thru 0x4e0ee40 (0xa041 bytes) 2024-09-17T21:46:05.561 INFO:teuthology.orchestra.run.smithi165.stdout:100 write 0x17ecc00 thru 0x17f6225 (0x9626 bytes) 2024-09-17T21:46:05.561 INFO:teuthology.orchestra.run.smithi165.stdout:102 write 0x33cb200 thru 0x33d03db (0x51dc bytes) 2024-09-17T21:46:05.566 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.566 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.577 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:05.583 INFO:teuthology.orchestra.run.smithi165.stdout:104 write 0x65e3800 thru 0x65ea5a2 (0x6da3 bytes) 2024-09-17T21:46:05.601 INFO:teuthology.orchestra.run.smithi165.stdout:105 write 0xc637200 thru 0xc63c12b (0x4f2c bytes) 2024-09-17T21:46:05.606 INFO:teuthology.orchestra.run.smithi165.stdout:109 punch from 0x44ae200 to 0x44b8e00, (0xac00 bytes) 2024-09-17T21:46:05.607 INFO:teuthology.orchestra.run.smithi165.stdout:110 read 0x6782e00 thru 0x6788e30 (0x6031 bytes) 2024-09-17T21:46:05.608 INFO:teuthology.orchestra.run.smithi165.stdout:113 read 0x5213800 thru 0x521d45e (0x9c5f bytes) 2024-09-17T21:46:05.608 INFO:teuthology.orchestra.run.smithi165.stdout:114 read 0xb885800 thru 0xb88689e (0x109f bytes) 2024-09-17T21:46:05.608 INFO:teuthology.orchestra.run.smithi165.stdout:115 write 0xc0bfa00 thru 0xc0c371e (0x3d1f bytes) 2024-09-17T21:46:05.608 INFO:teuthology.orchestra.run.smithi165.stdout:116 punch from 0x110b400 to 0x1118600, (0xd200 bytes) 2024-09-17T21:46:05.608 INFO:teuthology.orchestra.run.smithi165.stdout:119 punch from 0x9721000 to 0x9724a00, (0x3a00 bytes) 2024-09-17T21:46:05.609 INFO:teuthology.orchestra.run.smithi165.stdout:120 write 0x99e4e00 thru 0x99ebc3a (0x6e3b bytes) 2024-09-17T21:46:05.609 INFO:teuthology.orchestra.run.smithi165.stdout:121 read 0x8b21400 thru 0x8b291c0 (0x7dc1 bytes) 2024-09-17T21:46:05.609 INFO:teuthology.orchestra.run.smithi165.stdout:122 write 0x7787600 thru 0x7791336 (0x9d37 bytes) 2024-09-17T21:46:05.609 INFO:teuthology.orchestra.run.smithi165.stdout:126 read 0x810f600 thru 0x8110407 (0xe08 bytes) 2024-09-17T21:46:05.623 INFO:teuthology.orchestra.run.smithi165.stdout:127 write 0xe46de00 thru 0xe479cb2 (0xbeb3 bytes) 2024-09-17T21:46:05.628 INFO:teuthology.orchestra.run.smithi165.stdout:128 read 0x8e8400 thru 0x8f4188 (0xbd89 bytes) 2024-09-17T21:46:05.644 INFO:teuthology.orchestra.run.smithi165.stdout:130 trunc from 0xe479e00 to 0x2467400 2024-09-17T21:46:05.657 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:05.655+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:05.661 INFO:teuthology.orchestra.run.smithi165.stdout:132 punch from 0x21e3800 to 0x21f0400, (0xcc00 bytes) 2024-09-17T21:46:05.667 INFO:teuthology.orchestra.run.smithi165.stdout:135 read 0x2eea00 thru 0x2f2f1f (0x4520 bytes) 2024-09-17T21:46:05.684 INFO:teuthology.orchestra.run.smithi165.stdout:137 write 0x2f71200 thru 0x2f79788 (0x8589 bytes) 2024-09-17T21:46:05.703 INFO:teuthology.orchestra.run.smithi165.stdout:138 write 0x5dc9800 thru 0x5dcc300 (0x2b01 bytes) 2024-09-17T21:46:05.718 INFO:teuthology.orchestra.run.smithi165.stdout:139 write 0x8524200 thru 0x85268de (0x26df bytes) 2024-09-17T21:46:05.737 INFO:teuthology.orchestra.run.smithi165.stdout:142 write 0x9a06000 thru 0x9a064b0 (0x4b1 bytes) 2024-09-17T21:46:05.742 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.742 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.744 INFO:teuthology.orchestra.run.smithi165.stdout:143 write 0x842f800 thru 0x843578c (0x5f8d bytes) 2024-09-17T21:46:05.750 INFO:teuthology.orchestra.run.smithi165.stdout:147 trunc from 0x9a06600 to 0x4805200 2024-09-17T21:46:05.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:05.785 INFO:teuthology.orchestra.run.smithi165.stdout:149 write 0xbb12800 thru 0xbb1333a (0xb3b bytes) 2024-09-17T21:46:05.785 INFO:teuthology.orchestra.run.smithi165.stdout:154 write 0x57a0c00 thru 0x57ae383 (0xd784 bytes) 2024-09-17T21:46:05.785 INFO:teuthology.orchestra.run.smithi165.stdout:156 write 0x204cc00 thru 0x2058854 (0xbc55 bytes) 2024-09-17T21:46:05.787 INFO:teuthology.orchestra.run.smithi165.stdout:157 read 0xac85800 thru 0xac8f601 (0x9e02 bytes) 2024-09-17T21:46:05.787 INFO:teuthology.orchestra.run.smithi165.stdout:162 read 0x26e3a00 thru 0x26e618a (0x278b bytes) 2024-09-17T21:46:05.787 INFO:teuthology.orchestra.run.smithi165.stdout:163 write 0x4cd1000 thru 0x4cd1362 (0x363 bytes) 2024-09-17T21:46:05.796 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:05.793+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:05.807 INFO:teuthology.orchestra.run.smithi165.stdout:165 write 0xd100a00 thru 0xd10d64e (0xcc4f bytes) 2024-09-17T21:46:05.807 INFO:teuthology.orchestra.run.smithi165.stdout:166 read 0xa03b800 thru 0xa046d8b (0xb58c bytes) 2024-09-17T21:46:05.807 INFO:teuthology.orchestra.run.smithi165.stdout:168 read 0x75f2c00 thru 0x75f98bf (0x6cc0 bytes) 2024-09-17T21:46:05.807 INFO:teuthology.orchestra.run.smithi165.stdout:169 read 0x446b200 thru 0x446f42a (0x422b bytes) 2024-09-17T21:46:05.820 INFO:teuthology.orchestra.run.smithi165.stdout:172 trunc from 0xd10d800 to 0x1d40600 2024-09-17T21:46:05.846 INFO:teuthology.orchestra.run.smithi165.stdout:173 punch from 0x18bf800 to 0x18c7200, (0x7a00 bytes) 2024-09-17T21:46:05.859 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:05.856+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:05.868 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:05.868 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:05.874 INFO:teuthology.orchestra.run.smithi165.stdout:174 write 0xc789a00 thru 0xc79448b (0xaa8c bytes) 2024-09-17T21:46:05.874 INFO:teuthology.orchestra.run.smithi165.stdout:178 write 0x2f62800 thru 0x2f6acc8 (0x84c9 bytes) 2024-09-17T21:46:05.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:05.883 INFO:teuthology.orchestra.run.smithi165.stdout:180 trunc from 0xc794600 to 0x504d200 2024-09-17T21:46:05.902 INFO:teuthology.orchestra.run.smithi165.stdout:182 trunc from 0x504d200 to 0x3afa800 2024-09-17T21:46:05.928 INFO:teuthology.orchestra.run.smithi165.stdout:185 write 0xa5dfc00 thru 0xa5e857a (0x897b bytes) 2024-09-17T21:46:05.934 INFO:teuthology.orchestra.run.smithi165.stdout:truncating to largest ever: 0xea1f400 2024-09-17T21:46:05.939 INFO:teuthology.orchestra.run.smithi165.stdout:187 trunc from 0xa5e8600 to 0xea1f400 2024-09-17T21:46:05.952 INFO:teuthology.orchestra.run.smithi165.stdout:192 read 0x14f0200 thru 0x14ff449 (0xf24a bytes) 2024-09-17T21:46:05.958 INFO:teuthology.orchestra.run.smithi165.stdout:195 write 0x68ba600 thru 0x68bb4b1 (0xeb2 bytes) 2024-09-17T21:46:05.959 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:05.957+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:05.967 INFO:teuthology.orchestra.run.smithi165.stdout:197 trunc from 0xea1f400 to 0x79eb600 2024-09-17T21:46:05.998 INFO:teuthology.orchestra.run.smithi165.stdout:199 write 0xbb99000 thru 0xbb9c2aa (0x32ab bytes) 2024-09-17T21:46:06.007 INFO:teuthology.orchestra.run.smithi165.stdout:200 trunc from 0xbb9c400 to 0x829ac00 2024-09-17T21:46:06.034 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.034 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.042 INFO:teuthology.orchestra.run.smithi165.stdout:201 write 0xa5d5c00 thru 0xa5db6e5 (0x5ae6 bytes) 2024-09-17T21:46:06.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:06.048 INFO:teuthology.orchestra.run.smithi165.stdout:204 trunc from 0xa5db800 to 0xb8cb200 2024-09-17T21:46:06.061 INFO:teuthology.orchestra.run.smithi165.stdout:205 read 0x8469c00 thru 0x847684a (0xcc4b bytes) 2024-09-17T21:46:06.065 INFO:teuthology.orchestra.run.smithi165.stdout:206 write 0x92ac400 thru 0x92badb3 (0xe9b4 bytes) 2024-09-17T21:46:06.069 INFO:teuthology.orchestra.run.smithi165.stdout:207 trunc from 0xb8cb200 to 0xe0fe200 2024-09-17T21:46:06.080 INFO:teuthology.orchestra.run.smithi165.stdout:208 write 0xcb5ce00 thru 0xcb6ba84 (0xec85 bytes) 2024-09-17T21:46:06.086 INFO:teuthology.orchestra.run.smithi165.stdout:209 clone 1 order 23 su 2097152 sc 6 2024-09-17T21:46:06.088 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:06.085+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:06.156 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.156 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.161 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:06.159+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:06.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:06.262 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:06.260+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:06.289 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:06.290 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:06.290 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T21:46:06.290 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:46:06.290 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:46:06.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T21:46:06.298 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:46:06.322 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.322 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:06.374 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:06.371+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:06.399 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:46:06.399 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:06.465 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:06.462+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:06.482 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.482 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:06.566 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:06.563+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:06.600 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.600 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:06.622 INFO:teuthology.orchestra.run.smithi165.stdout:210 trunc from 0xe0fe200 to 0xbc8be00 2024-09-17T21:46:06.638 INFO:teuthology.orchestra.run.smithi165.stdout:211 punch from 0x7c25400 to 0x7c2dc00, (0x8800 bytes) 2024-09-17T21:46:06.640 INFO:teuthology.orchestra.run.smithi165.stdout:214 read 0x56a3c00 thru 0x56a6c47 (0x3048 bytes) 2024-09-17T21:46:06.640 INFO:teuthology.orchestra.run.smithi165.stdout:216 punch from 0x1eee400 to 0x1ef8c00, (0xa800 bytes) 2024-09-17T21:46:06.641 INFO:teuthology.orchestra.run.smithi165.stdout:218 read 0xa01da00 thru 0xa02b460 (0xda61 bytes) 2024-09-17T21:46:06.641 INFO:teuthology.orchestra.run.smithi165.stdout:219 read 0x6542200 thru 0x6545c49 (0x3a4a bytes) 2024-09-17T21:46:06.641 INFO:teuthology.orchestra.run.smithi165.stdout:221 read 0x85bd200 thru 0x85c58e9 (0x86ea bytes) 2024-09-17T21:46:06.641 INFO:teuthology.orchestra.run.smithi165.stdout:222 write 0x9111000 thru 0x911de6b (0xce6c bytes) 2024-09-17T21:46:06.641 INFO:teuthology.orchestra.run.smithi165.stdout:223 write 0x25d0000 thru 0x25db045 (0xb046 bytes) 2024-09-17T21:46:06.642 INFO:teuthology.orchestra.run.smithi165.stdout:224 punch from 0x1193600 to 0x11a1400, (0xde00 bytes) 2024-09-17T21:46:06.642 INFO:teuthology.orchestra.run.smithi165.stdout:226 read 0x25b5c00 thru 0x25bb7fa (0x5bfb bytes) 2024-09-17T21:46:06.642 INFO:teuthology.orchestra.run.smithi165.stdout:227 write 0x9972a00 thru 0x997395d (0xf5e bytes) 2024-09-17T21:46:06.643 INFO:teuthology.orchestra.run.smithi165.stdout:228 trunc from 0xbc8be00 to 0xb0aa600 2024-09-17T21:46:06.659 INFO:teuthology.orchestra.run.smithi165.stdout:230 read 0x22c8400 thru 0x22d494c (0xc54d bytes) 2024-09-17T21:46:06.666 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:06.663+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:06.669 INFO:teuthology.orchestra.run.smithi165.stdout:231 trunc from 0xb0aa600 to 0x34ee400 2024-09-17T21:46:06.702 INFO:teuthology.orchestra.run.smithi165.stdout:232 trunc from 0x34ee400 to 0xd2af600 2024-09-17T21:46:06.711 INFO:teuthology.orchestra.run.smithi165.stdout:233 read 0x8a3aa00 thru 0x8a3d9ee (0x2fef bytes) 2024-09-17T21:46:06.717 INFO:teuthology.orchestra.run.smithi165.stdout:235 read 0xaa0f800 thru 0xaa1cd9f (0xd5a0 bytes) 2024-09-17T21:46:06.717 INFO:teuthology.orchestra.run.smithi165.stdout:237 write 0x6998e00 thru 0x69a3a7c (0xac7d bytes) 2024-09-17T21:46:06.717 INFO:teuthology.orchestra.run.smithi165.stdout:240 read 0x2826000 thru 0x282a147 (0x4148 bytes) 2024-09-17T21:46:06.718 INFO:teuthology.orchestra.run.smithi165.stdout:247 read 0x86bde00 thru 0x86ca634 (0xc835 bytes) 2024-09-17T21:46:06.718 INFO:teuthology.orchestra.run.smithi165.stdout:248 punch from 0x65ece00 to 0x65fa400, (0xd600 bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:249 punch from 0x7e40a00 to 0x7e4a400, (0x9a00 bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:250 write 0x6e74600 thru 0x6e8217f (0xdb80 bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:251 write 0xccfba00 thru 0xcd0347a (0x7a7b bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:252 read 0x5c67400 thru 0x5c6ea5e (0x765f bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:254 punch from 0xcd2e00 to 0xcdd400, (0xa600 bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:256 punch from 0x714ce00 to 0x7150200, (0x3400 bytes) 2024-09-17T21:46:06.719 INFO:teuthology.orchestra.run.smithi165.stdout:257 read 0x1a67200 thru 0x1a7634d (0xf14e bytes) 2024-09-17T21:46:06.720 INFO:teuthology.orchestra.run.smithi165.stdout:259 punch from 0x47b6c00 to 0x47c0e00, (0xa200 bytes) 2024-09-17T21:46:06.723 INFO:teuthology.orchestra.run.smithi165.stdout:261 trunc from 0xd2af600 to 0xa152e00 2024-09-17T21:46:06.739 INFO:teuthology.orchestra.run.smithi165.stdout:263 read 0x1e60c00 thru 0x1e70487 (0xf888 bytes) 2024-09-17T21:46:06.758 INFO:teuthology.orchestra.run.smithi165.stdout:264 write 0xd7fd600 thru 0xd809b20 (0xc521 bytes) 2024-09-17T21:46:06.758 INFO:teuthology.orchestra.run.smithi165.stdout:265 clone 2 order 24 su 1048576 sc 10 2024-09-17T21:46:06.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:06.763 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.763 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.766 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:06.764+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:06.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:06.868 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:06.865+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:06.883 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:06.884 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:06.900 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:06.901 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:06.917 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:06.917 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:06.919 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:06.920 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:06.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:06.933 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:06.933 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:06.950 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:06.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:06.967 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:06.968 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:06.969 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:06.966+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:07.036 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:07.037 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.045 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:07.070 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:07.067+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:07.151 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.151 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:07.151 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:07.151 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:07.151 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:07.171 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:07.168+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9904938890000001, 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036959290000000001, 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274238+0000", 2024-09-17T21:46:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.829e-06 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274245+0000", 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1810000000000004e-06 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274262+0000", 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6239000000000002e-05 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274270+0000", 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5399999999999996e-06 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274341+0000", 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1357000000000001e-05 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274349+0000", 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7100000000000007e-06 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276472+0000", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021225319999999999 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276559+0000", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7515999999999994e-05 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276665+0000", 2024-09-17T21:46:07.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105621 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277452+0000", 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078702500000000001 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277862+0000", 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041006899999999999 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277899+0000", 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7373000000000003e-05 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277932+0000", 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2937e-05 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4370.0:9 3.1a e23/21 ondisk, result = 0)", 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9869600549999999, 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000168856, 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277770+0000", 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277771+0000", 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8599999999999997e-07 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277776+0000", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277787+0000", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0703000000000001e-05 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277794+0000", 2024-09-17T21:46:07.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3719999999999999e-06 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277847+0000", 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2476999999999999e-05 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277855+0000", 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3650000000000004e-06 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277939+0000", 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3912999999999994e-05 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:10 3.1c e24/21 3:3831950c:::rbd_directory:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9825813180000003, 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011563879999999999, 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282149+0000", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282155+0000", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4500000000000003e-06 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282165+0000", 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0572e-05 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282179+0000", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4151999999999999e-05 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282246+0000", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6095999999999999e-05 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.282273+0000", 2024-09-17T21:46:07.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7549999999999999e-05 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283249+0000", 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000975671 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283305+0000", 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6499999999999998e-05 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:12 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9783711779999997, 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061919899999999996, 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286359+0000", 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286364+0000", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5260000000000004e-06 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286380+0000", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5733999999999999e-05 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286388+0000", 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4950000000000008e-06 2024-09-17T21:46:07.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286447+0000", 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8798999999999997e-05 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286461+0000", 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4576000000000001e-05 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286939+0000", 2024-09-17T21:46:07.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047777699999999998 2024-09-17T21:46:07.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.286978+0000", 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9011999999999999e-05 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:13 3.1 e24/21 3:83138b25:::rbd_object_map.111282762508:head v 24'1, mlcod=0'0)", 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.97683901, 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093130700000000003, 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287891+0000", 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287892+0000", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287896+0000", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6229999999999996e-06 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287907+0000", 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0346000000000001e-05 2024-09-17T21:46:07.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287913+0000", 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.896e-06 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.287980+0000", 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7084999999999999e-05 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288002+0000", 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2112000000000001e-05 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288776+0000", 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077369400000000001 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.288822+0000", 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6940999999999997e-05 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:23 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9674186090000001, 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000328468, 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:07.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 23 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297312+0000", 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297313+0000", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.792e-06 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297320+0000", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8719999999999996e-06 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297334+0000", 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4151999999999999e-05 2024-09-17T21:46:07.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297341+0000", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6780000000000003e-06 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297408+0000", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6873999999999997e-05 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297508+0000", 2024-09-17T21:46:07.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9580000000000005e-05 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.297640+0000", 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013252000000000001 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4370.0:25 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'2, mlcod=0'0)", 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:07.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.965601361, 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061023999999999996, 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299129+0000", 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299130+0000", 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2900000000000002e-07 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299136+0000", 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8850000000000004e-06 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299148+0000", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2218999999999999e-05 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299163+0000", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5167000000000001e-05 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299214+0000", 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1368e-05 2024-09-17T21:46:07.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299230+0000", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5971e-05 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299701+0000", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047061 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.299739+0000", 2024-09-17T21:46:07.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8191000000000001e-05 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4455.0:10 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.819092988, 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032502799999999999, 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4455", 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2139432296", 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445637+0000", 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445638+0000", 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.04e-06 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445643+0000", 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0409999999999997e-06 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445652+0000", 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9099999999999994e-06 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445659+0000", 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6019999999999996e-06 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445727+0000", 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8429999999999994e-05 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445827+0000", 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010010599999999999 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.445962+0000", 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013489900000000001 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4455.0:12 3.1b e24/21 3:ddaf79d8:::rbd_header.111282762508:head v 24'3, mlcod=24'1)", 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.817672489, 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058209400000000001, 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:07.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447058+0000", 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447059+0000", 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0100000000000001e-06 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447067+0000", 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2269999999999993e-06 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447080+0000", 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3291e-05 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447086+0000", 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.071e-06 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447136+0000", 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9484999999999998e-05 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447154+0000", 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7827999999999999e-05 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447607+0000", 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045376599999999998 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.447640+0000", 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2416000000000003e-05 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4455.0:13 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:07.313 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7808482960000003, 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033991400000000003, 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4455", 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2139432296", 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483882+0000", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483883+0000", 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2500000000000001e-06 2024-09-17T21:46:07.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483889+0000", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8660000000000002e-06 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483903+0000", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4428e-05 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483909+0000", 2024-09-17T21:46:07.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0750000000000003e-06 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.483992+0000", 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2095999999999995e-05 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.484087+0000", 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4988999999999995e-05 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.484222+0000", 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013521 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.318 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.320 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:46:07.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:07.321 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:07.317+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:07.362 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone1 from 0xd809c00 (overlap 0x34ee400) to 0x1a77200 2024-09-17T21:46:07.420 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:07.417+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:07.433 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:07.434 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:07.521 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:07.519+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:07.592 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:07.592 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:07.645 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:07.643+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:07.666 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.714 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.474647761, 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037970529999999999, 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234864+0000", 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2342e-05 2024-09-17T21:46:07.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234874+0000", 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0078000000000001e-05 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234894+0000", 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9208e-05 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234908+0000", 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4255e-05 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235001+0000", 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3245000000000006e-05 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:07.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235014+0000", 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2692e-05 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236563+0000", 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001549258 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236725+0000", 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016205000000000001 2024-09-17T21:46:07.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236884+0000", 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015916699999999999 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.237894+0000", 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010101579999999999 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238559+0000", 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000664483 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238595+0000", 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5867000000000001e-05 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238649+0000", 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4249999999999997e-05 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.472860432999999, 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034029759999999998, 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:07.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236642+0000", 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7020000000000001e-06 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236654+0000", 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2207e-05 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236674+0000", 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9497e-05 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236688+0000", 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.397e-05 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236772+0000", 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4654000000000006e-05 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236780+0000", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9629999999999998e-06 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238360+0000", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001579843 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238517+0000", 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015724700000000001 2024-09-17T21:46:07.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238676+0000", 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015822000000000001 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239625+0000", 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094962200000000001 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239928+0000", 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000302684 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239992+0000", 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4579000000000006e-05 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.240042+0000", 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9787999999999998e-05 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.728 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e19)", 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:54.242130+0000", 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.467369439, 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014797479999999999, 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:07.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 7 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242130+0000", 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242130+0000", 2024-09-17T21:46:07.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242132+0000", 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.64e-06 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242139+0000", 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242154+0000", 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4637999999999999e-05 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242162+0000", 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.994e-06 2024-09-17T21:46:07.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242223+0000", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1235000000000006e-05 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242295+0000", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2228e-05 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.242403+0000", 2024-09-17T21:46:07.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108041 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.243065+0000", 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066233899999999996 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.243544+0000", 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047812100000000003 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.243579+0000", 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5144000000000001e-05 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:54.243610+0000", 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1285000000000003e-05 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4301961270000003, 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042466919999999998, 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:07.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:07.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279305+0000", 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.339e-06 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279310+0000", 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279317+0000", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5059999999999998e-06 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279325+0000", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999996e-06 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279393+0000", 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8181000000000003e-05 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279400+0000", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281620+0000", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022195130000000002 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281751+0000", 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013134099999999999 2024-09-17T21:46:07.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281966+0000", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021533199999999999 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283188+0000", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001221895 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283464+0000", 2024-09-17T21:46:07.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027606000000000001 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283508+0000", 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4026000000000002e-05 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283550+0000", 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1959000000000003e-05 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:855 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.328843+0000", 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.38065644399999998, 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001136169, 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:07.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 855 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328843+0000", 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328843+0000", 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328844+0000", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.95e-07 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328850+0000", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3870000000000003e-06 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328862+0000", 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2122e-05 2024-09-17T21:46:07.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328870+0000", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9109999999999993e-06 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328929+0000", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9404999999999998e-05 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.328998+0000", 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8918999999999995e-05 2024-09-17T21:46:07.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.329138+0000", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014034099999999999 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.329692+0000", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055415100000000001 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.329906+0000", 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021383 2024-09-17T21:46:07.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.329939+0000", 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2747999999999999e-05 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.329979+0000", 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.036e-05 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:859 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.335515+0000", 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.37398505500000001, 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00144403, 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 859 2024-09-17T21:46:07.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335515+0000", 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335515+0000", 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335516+0000", 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7000000000000003e-07 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335522+0000", 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9989999999999997e-06 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335533+0000", 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1667e-05 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335538+0000", 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2209999999999996e-06 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335595+0000", 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6811999999999997e-05 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335739+0000", 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014368399999999999 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.335938+0000", 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000198555 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.336655+0000", 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071740499999999995 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.336889+0000", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000233897 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.336922+0000", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2737999999999997e-05 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.336959+0000", 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7082000000000002e-05 2024-09-17T21:46:07.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:587 3.c 3:32a591c3:::rbd_data.1112ff4590f3.0000000000000010:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.34374126799999999, 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025837009999999999, 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:07.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 587 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365760+0000", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9599999999999999e-06 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365768+0000", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8080000000000005e-06 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365795+0000", 2024-09-17T21:46:07.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6339e-05 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365805+0000", 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9739999999999997e-06 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365877+0000", 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2228e-05 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365979+0000", 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010268999999999999 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366179+0000", 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199314 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367176+0000", 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0009969200000000001 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368214+0000", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010379370000000001 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368279+0000", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5061999999999995e-05 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368342+0000", 2024-09-17T21:46:07.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3468999999999998e-05 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:605 3.c 3:362cbce7:::rbd_data.111282762508.0000000000000006:3 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.371486+0000", 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33801346300000001, 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058139599999999997, 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 605 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371486+0000", 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371486+0000", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371487+0000", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5400000000000001e-07 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371490+0000", 2024-09-17T21:46:07.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4709999999999999e-06 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371499+0000", 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5629999999999992e-06 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371509+0000", 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0028e-05 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371836+0000", 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000327153 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371931+0000", 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5389000000000004e-05 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372068+0000", 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013623800000000001 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:611 3.e 3:713ae996:::rbd_data.111282762508.000000000000000b:3 [sparse-read 2097152~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.371601+0000", 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33789897099999999, 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00060181099999999995, 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:07.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 611 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371601+0000", 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371601+0000", 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371602+0000", 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9600000000000008e-07 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371606+0000", 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1540000000000004e-06 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371677+0000", 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0753000000000005e-05 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371685+0000", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4279999999999996e-06 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371832+0000", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014659100000000001 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371917+0000", 2024-09-17T21:46:07.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5124999999999996e-05 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372203+0000", 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028576399999999998 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4465.0:618 3.2 e27/21 3:41bbdce2:::rbd_data.1112ff4590f3.0000000000000003:head v 27'6, mlcod=26'3)", 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.375273+0000", 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.334226996, 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012872960000000001, 2024-09-17T21:46:07.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375273+0000", 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375273+0000", 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375273+0000", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4799999999999999e-07 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375279+0000", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2319999999999998e-06 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375298+0000", 2024-09-17T21:46:07.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8474999999999998e-05 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375308+0000", 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0180000000000001e-05 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375361+0000", 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3001999999999998e-05 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375383+0000", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1868000000000001e-05 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376519+0000", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011358570000000001 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376560+0000", 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1334000000000001e-05 2024-09-17T21:46:07.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4465.0:621 3.2 e27/21 3:47bde0a9:::rbd_data.1112ff4590f3.0000000000000005:head v 27'7, mlcod=26'3)", 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.376817+0000", 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33268302, 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001100204, 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376817+0000", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376817+0000", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376817+0000", 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5400000000000001e-07 2024-09-17T21:46:07.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376935+0000", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011766500000000001 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376954+0000", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8904999999999999e-05 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376960+0000", 2024-09-17T21:46:07.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1929999999999998e-06 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376998+0000", 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7864000000000002e-05 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377010+0000", 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1986e-05 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377869+0000", 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085907599999999996 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377917+0000", 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7960999999999997e-05 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4465.0:622 3.2 e27/21 3:47bde0a9:::rbd_data.1112ff4590f3.0000000000000005:head v 27'9, mlcod=26'3)", 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.377436+0000", 2024-09-17T21:46:07.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.332063416, 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070163199999999999, 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377436+0000", 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377436+0000", 2024-09-17T21:46:07.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377437+0000", 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377442+0000", 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0749999999999997e-06 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377449+0000", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7889999999999997e-06 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377466+0000", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7713000000000001e-05 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377507+0000", 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1097000000000003e-05 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.377520+0000", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2874000000000001e-05 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.378104+0000", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058361600000000002 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.378138+0000", 2024-09-17T21:46:07.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4028e-05 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:19 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 1536~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.661010+0000", 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.048489519000000002, 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047731599999999998, 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661010+0000", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661010+0000", 2024-09-17T21:46:07.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661011+0000", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1399999999999997e-07 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661014+0000", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4529999999999998e-06 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661021+0000", 2024-09-17T21:46:07.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9229999999999996e-06 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661026+0000", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4160000000000003e-06 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661393+0000", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036688999999999998 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661435+0000", 2024-09-17T21:46:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1696000000000002e-05 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661487+0000", 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2423999999999998e-05 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:20 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 2048~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.661034+0000", 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.048465844000000001, 2024-09-17T21:46:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057141699999999995, 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661034+0000", 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661034+0000", 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661034+0000", 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661036+0000", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661042+0000", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0480000000000004e-06 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661048+0000", 2024-09-17T21:46:07.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3720000000000004e-06 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661511+0000", 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046330900000000002 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661552+0000", 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0481000000000001e-05 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661605+0000", 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3733000000000002e-05 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:21 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 2560~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.661057+0000", 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.048442491999999997, 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066321800000000003, 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 21 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661057+0000", 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661057+0000", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661058+0000", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000002e-07 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661069+0000", 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1178999999999999e-05 2024-09-17T21:46:07.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661076+0000", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3699999999999997e-06 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661082+0000", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6760000000000004e-06 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661628+0000", 2024-09-17T21:46:07.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054566700000000001 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661666+0000", 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8705999999999998e-05 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661720+0000", 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4172999999999998e-05 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:22 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 3072~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.661095+0000", 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.048404770999999999, 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074274699999999998, 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661095+0000", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661095+0000", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661095+0000", 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2600000000000002e-07 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661098+0000", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0110000000000001e-06 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661107+0000", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7539999999999995e-06 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661116+0000", 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661746+0000", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062999900000000001 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661784+0000", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7932000000000002e-05 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661838+0000", 2024-09-17T21:46:07.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3792000000000003e-05 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:23 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 3584~512] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.661134+0000", 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.048365711999999998, 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081224599999999997, 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 23 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661134+0000", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661134+0000", 2024-09-17T21:46:07.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661135+0000", 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7899999999999998e-07 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661137+0000", 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4609999999999998e-06 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661153+0000", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6470999999999999e-05 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661163+0000", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9550000000000004e-06 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661852+0000", 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068847600000000004 2024-09-17T21:46:07.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661889+0000", 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7038e-05 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.661946+0000", 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7265999999999997e-05 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:55 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 69632~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.672415+0000", 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.037084944000000002, 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042335000000000002, 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 55 2024-09-17T21:46:07.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672415+0000", 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672415+0000", 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672415+0000", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7800000000000002e-07 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672426+0000", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1316e-05 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672433+0000", 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9260000000000003e-06 2024-09-17T21:46:07.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672439+0000", 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1449999999999999e-06 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672731+0000", 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029249400000000002 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672770+0000", 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8518e-05 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672838+0000", 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8572999999999996e-05 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:56 3.c 3:331ce048:::rbd_data.111282762508.0000000000000000:head [read 135168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.672449+0000", 2024-09-17T21:46:07.811 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.037050240999999998, 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050580299999999998, 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 56 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672449+0000", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672449+0000", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672450+0000", 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:46:07.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672452+0000", 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672459+0000", 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4529999999999999e-06 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672464+0000", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9609999999999996e-06 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672857+0000", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039316299999999998 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672895+0000", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8402999999999998e-05 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.672955+0000", 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9987e-05 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4427.0:66 3.1d 3:bec6f9a9:::rbd_data.111282762508.0000000000000038:head [sparse-read 577536~77824] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.678205+0000", 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.031294303000000002, 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044768100000000001, 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:07.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4427", 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 66 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678205+0000", 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678205+0000", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678206+0000", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678214+0000", 2024-09-17T21:46:07.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6149999999999997e-06 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678222+0000", 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7149999999999998e-06 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:07.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678227+0000", 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2970000000000003e-06 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678562+0000", 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000334087 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.678653+0000", 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1497999999999997e-05 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:07.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:07.821 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:07.822 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:07.819+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:07.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:07.922 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:07.920+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:07.938 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:07.938 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:07.938 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:07.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:08.023 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:08.021+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:08.060 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:08.060 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:08.060 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:08.060 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:08.060 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:08.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:08.124 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:08.122+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:08.218 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:08.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:20 1.0 e14/13 1:30306672:devicehealth::main.db.0000000000000000:head v 14'10, mlcod=14'8)", 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.941981191, 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001138242, 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273177+0000", 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273178+0000", 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1299999999999999e-07 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273284+0000", 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106878 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273298+0000", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3602e-05 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273304+0000", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6500000000000001e-06 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273356+0000", 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2391e-05 2024-09-17T21:46:08.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.273378+0000", 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1729000000000002e-05 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.274268+0000", 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089036499999999995 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.274315+0000", 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6814000000000003e-05 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:31 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'17, mlcod=14'14)", 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.933231353, 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001134208, 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281927+0000", 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5000000000000002e-07 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281935+0000", 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2749999999999998e-06 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281949+0000", 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4667999999999999e-05 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.281963+0000", 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4069e-05 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282001+0000", 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7839000000000001e-05 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282016+0000", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4304000000000001e-05 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283000+0000", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00098423299999999994 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283061+0000", 2024-09-17T21:46:08.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1069999999999996e-05 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:32 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'18, mlcod=14'14)", 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.932730942999999, 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001232489, 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282427+0000", 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282428+0000", 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6000000000000003e-07 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282501+0000", 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3473000000000006e-05 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282513+0000", 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1942e-05 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282519+0000", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282571+0000", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1884e-05 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.282591+0000", 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9576e-05 2024-09-17T21:46:08.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283611+0000", 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001020389 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283660+0000", 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8433000000000002e-05 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:34 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'20, mlcod=14'16)", 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.931266750999999, 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001166865, 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:08.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283891+0000", 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283892+0000", 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283897+0000", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4609999999999999e-06 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283911+0000", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3565e-05 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283917+0000", 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283967+0000", 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0114999999999997e-05 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.283996+0000", 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8592e-05 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285023+0000", 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010272250000000001 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285058+0000", 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5238999999999999e-05 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:35 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'21, mlcod=14'16)", 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:08.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.930734083000001, 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012645860000000001, 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284424+0000", 2024-09-17T21:46:08.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284425+0000", 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.68e-07 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284504+0000", 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9231000000000001e-05 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284518+0000", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4185000000000001e-05 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284524+0000", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.519e-06 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284583+0000", 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9759999999999997e-05 2024-09-17T21:46:08.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.284607+0000", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3305000000000001e-05 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285654+0000", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010475840000000001 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.285688+0000", 2024-09-17T21:46:08.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4334e-05 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:55 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'33, mlcod=14'29)", 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.914782241000001, 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001273038, 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300376+0000", 2024-09-17T21:46:08.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300467+0000", 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1119999999999995e-05 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300478+0000", 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0896999999999999e-05 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300490+0000", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1747e-05 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300641+0000", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015110300000000001 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300671+0000", 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0215000000000001e-05 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301605+0000", 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093397900000000004 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301649+0000", 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3377999999999997e-05 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.247 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:56 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'34, mlcod=14'29)", 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.914397976, 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012141910000000001, 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300760+0000", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300761+0000", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300765+0000", 2024-09-17T21:46:08.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6890000000000003e-06 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300777+0000", 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.300783+0000", 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4910000000000002e-06 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301285+0000", 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050200500000000003 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301309+0000", 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3451e-05 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301943+0000", 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063448900000000002 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.301974+0000", 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0929999999999997e-05 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4189.0:137 1.0 e14/13 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 14'95, mlcod=14'90)", 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.392257514000001, 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012645969999999999, 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822900+0000", 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822901+0000", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2e-07 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822958+0000", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6549999999999999e-05 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822969+0000", 2024-09-17T21:46:08.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1892999999999999e-05 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.822974+0000", 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1509999999999997e-06 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.823016+0000", 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2521000000000002e-05 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.823036+0000", 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0259e-05 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.824121+0000", 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010846370000000001 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:45:47.824165+0000", 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4066000000000001e-05 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:03.283989+0000", 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.9311685279999997, 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015423979999999999, 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.283989+0000", 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.283989+0000", 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.283991+0000", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284002+0000", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1729000000000001e-05 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284015+0000", 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2665e-05 2024-09-17T21:46:08.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284023+0000", 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7109999999999995e-06 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284076+0000", 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000001e-05 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284177+0000", 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010038400000000001 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284320+0000", 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014331700000000001 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.284933+0000", 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061315100000000004 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285464+0000", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053138199999999995 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285501+0000", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6542999999999999e-05 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285532+0000", 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0926000000000002e-05 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:03.285836+0000", 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.9293222910000001, 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013672510000000001, 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.263 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285836+0000", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285836+0000", 2024-09-17T21:46:08.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285837+0000", 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2999999999999999e-07 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285842+0000", 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-06 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285852+0000", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0713000000000001e-05 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285860+0000", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7859999999999997e-06 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.285913+0000", 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2911000000000002e-05 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.286014+0000", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100704 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.286238+0000", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000223699 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.286760+0000", 2024-09-17T21:46:08.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052281300000000001 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287107+0000", 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034662499999999998 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287156+0000", 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8522000000000001e-05 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287203+0000", 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7407999999999997e-05 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:13 3.1 3:83138b25:::rbd_object_map.111282762508:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:03.287514+0000", 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.9276435660000004, 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001566336, 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287514+0000", 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287514+0000", 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287515+0000", 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287519+0000", 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4209999999999997e-06 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287529+0000", 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287536+0000", 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9700000000000002e-06 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287598+0000", 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2104e-05 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287690+0000", 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1818999999999996e-05 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.287802+0000", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112816 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.288318+0000", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051569099999999998 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.288980+0000", 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066140600000000002 2024-09-17T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.289035+0000", 2024-09-17T21:46:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5418999999999999e-05 2024-09-17T21:46:08.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.289081+0000", 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5843999999999997e-05 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:25 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch watch cookie 140714604257456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:03.298581+0000", 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.9165771239999998, 2024-09-17T21:46:08.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001461457, 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 25 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298581+0000", 2024-09-17T21:46:08.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298581+0000", 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298582+0000", 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.271e-06 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298594+0000", 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2159e-05 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298609+0000", 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4973e-05 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298618+0000", 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2240000000000001e-06 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298677+0000", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8335000000000003e-05 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298768+0000", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0796000000000003e-05 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.298943+0000", 2024-09-17T21:46:08.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017507800000000001 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.299554+0000", 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061090999999999999 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.299905+0000", 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035147800000000001 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.299965+0000", 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9925e-05 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.300042+0000", 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7307999999999999e-05 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4455.0:12 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch watch cookie 139813265168464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:03.446764+0000", 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7683943930000003, 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001068069, 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4455", 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2139432296", 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.283 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446764+0000", 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446764+0000", 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446764+0000", 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446768+0000", 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3950000000000001e-06 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446776+0000", 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0239999999999997e-06 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446780+0000", 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7260000000000002e-06 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446816+0000", 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5377999999999999e-05 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446864+0000", 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8698e-05 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.446970+0000", 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105037 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.447462+0000", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049250100000000001 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.447764+0000", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030179600000000001 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.447793+0000", 2024-09-17T21:46:08.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.955e-05 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:03.447832+0000", 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8293000000000001e-05 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:841 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.289 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.317831+0000", 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.89732717299999998, 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022783170000000002, 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 841 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317831+0000", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317831+0000", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317834+0000", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8059999999999999e-06 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317851+0000", 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7093000000000001e-05 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317886+0000", 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5312999999999999e-05 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.317901+0000", 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5400000000000002e-05 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.318006+0000", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104187 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.318010+0000", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8570000000000003e-06 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.319808+0000", 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001797575 2024-09-17T21:46:08.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.319895+0000", 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7258999999999999e-05 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.320109+0000", 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021382700000000001 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4465.0:587 3.c e27/21 3:32a591c3:::rbd_data.1112ff4590f3.0000000000000010:head v 27'11, mlcod=26'8)", 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366549+0000", 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84860902100000002, 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015277520000000001, 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366549+0000", 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366549+0000", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366550+0000", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5899999999999996e-07 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366556+0000", 2024-09-17T21:46:08.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8410000000000002e-06 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366566+0000", 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5440000000000004e-06 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366576+0000", 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7550000000000006e-06 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366629+0000", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2864000000000001e-05 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366647+0000", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8110000000000001e-05 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368034+0000", 2024-09-17T21:46:08.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001386766 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368077+0000", 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3213000000000001e-05 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:963 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch watch cookie 140714604291440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.583271+0000", 2024-09-17T21:46:08.299 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.63188720200000004, 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001222051, 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 963 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583271+0000", 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583271+0000", 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583272+0000", 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2399999999999997e-07 2024-09-17T21:46:08.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583277+0000", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583297+0000", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9204000000000001e-05 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583302+0000", 2024-09-17T21:46:08.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2859999999999999e-06 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583362+0000", 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0544000000000002e-05 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583445+0000", 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2607000000000004e-05 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.583599+0000", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000154015 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.584061+0000", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046179800000000003 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.584425+0000", 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00036451099999999998 2024-09-17T21:46:08.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.584454+0000", 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9111999999999998e-05 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.584493+0000", 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8380000000000002e-05 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4427.0:101 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch reconnect cookie 140360873488624 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.724973+0000", 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.49018472699999999, 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012265729999999999, 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4427", 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 101 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724973+0000", 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724973+0000", 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724974+0000", 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724976+0000", 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.379e-06 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724983+0000", 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0600000000000002e-06 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.724987+0000", 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0489999999999997e-06 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.725027+0000", 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9409000000000002e-05 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.725069+0000", 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2211e-05 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.726200+0000", 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011310249999999999 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4427.0:1978 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch unwatch cookie 140360873488624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.091822+0000", 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.12333596600000001, 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013361149999999999, 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4427", 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2866608967", 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1978 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091822+0000", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091822+0000", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091823+0000", 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2899999999999999e-06 2024-09-17T21:46:08.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091830+0000", 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9759999999999998e-06 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091845+0000", 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4902000000000001e-05 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091852+0000", 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5180000000000001e-06 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091927+0000", 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5111000000000006e-05 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.091990+0000", 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3337999999999996e-05 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.092092+0000", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010157000000000001 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.092723+0000", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00063134300000000003 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.093019+0000", 2024-09-17T21:46:08.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000296194 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.093080+0000", 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0455999999999999e-05 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.093158+0000", 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8417000000000005e-05 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:964 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch unwatch cookie 140714604291440] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.098157+0000", 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.117000912, 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001110555, 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 964 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098157+0000", 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098157+0000", 2024-09-17T21:46:08.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098158+0000", 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8400000000000004e-07 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098161+0000", 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098168+0000", 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6900000000000003e-06 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098173+0000", 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8879999999999996e-06 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098215+0000", 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2333999999999998e-05 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098261+0000", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5631999999999999e-05 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098340+0000", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8813999999999995e-05 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098690+0000", 2024-09-17T21:46:08.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034999899999999998 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.099185+0000", 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049488899999999998 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.099215+0000", 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0781999999999998e-05 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.099268+0000", 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2346000000000003e-05 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:965 3.1b 3:ddaf79d8:::rbd_header.111282762508:head [watch reconnect cookie 140714604291440 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.098185+0000", 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.116973151, 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018566730000000001, 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 965 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098185+0000", 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098185+0000", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098185+0000", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999998e-07 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098187+0000", 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:46:08.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098192+0000", 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9069999999999998e-06 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098195+0000", 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098525+0000", 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00033066900000000001 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.098611+0000", 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6002999999999996e-05 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.099298+0000", 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068648699999999995 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.099351+0000", 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3331000000000002e-05 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.100042+0000", 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069030800000000005 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.329 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:08.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:267 read 0xbe0d000 thru 0xbe15bb4 (0x8bb5 bytes) 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:268 write 0xdfcc200 thru 0xdfd0320 (0x4121 bytes) 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:269 write 0x1731200 thru 0x1735a8a (0x488b bytes) 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:270 read 0x6384a00 thru 0x6394882 (0xfe83 bytes) 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:271 read 0xd59e00 thru 0xd676ed (0xd8ee bytes) 2024-09-17T21:46:08.332 INFO:teuthology.orchestra.run.smithi165.stdout:272 write 0xa462200 thru 0xa46cb17 (0xa918 bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:273 punch from 0xb217a00 to 0xb219200, (0x1800 bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:274 punch from 0x955ce00 to 0x956a000, (0xd200 bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:275 read 0xc038800 thru 0xc042055 (0x9856 bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:279 trunc from 0xdfd0400 to 0x80d1400 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:283 read 0x4507000 thru 0x4508c6a (0x1c6b bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:284 read 0x6703400 thru 0x670e615 (0xb216 bytes) 2024-09-17T21:46:08.333 INFO:teuthology.orchestra.run.smithi165.stdout:285 trunc from 0x80d1400 to 0x7199600 2024-09-17T21:46:08.334 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:08.327+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:08.344 INFO:teuthology.orchestra.run.smithi165.stdout:286 read 0x3d31e00 thru 0x3d34248 (0x2449 bytes) 2024-09-17T21:46:08.353 INFO:teuthology.orchestra.run.smithi165.stdout:287 trunc from 0x7199600 to 0xe132e00 2024-09-17T21:46:08.363 INFO:teuthology.orchestra.run.smithi165.stdout:288 write 0x581ae00 thru 0x5820bac (0x5dad bytes) 2024-09-17T21:46:08.369 INFO:teuthology.orchestra.run.smithi165.stdout:289 read 0x130e400 thru 0x131cae2 (0xe6e3 bytes) 2024-09-17T21:46:08.369 INFO:teuthology.orchestra.run.smithi165.stdout:290 read 0xbf6200 thru 0xc03b69 (0xd96a bytes) 2024-09-17T21:46:08.370 INFO:teuthology.orchestra.run.smithi165.stdout:291 read 0x1279400 thru 0x127eb8a (0x578b bytes) 2024-09-17T21:46:08.370 INFO:teuthology.orchestra.run.smithi165.stdout:292 write 0x52f3000 thru 0x52fc57d (0x957e bytes) 2024-09-17T21:46:08.371 INFO:teuthology.orchestra.run.smithi165.stdout:295 write 0x1b15200 thru 0x1b1f74d (0xa54e bytes) 2024-09-17T21:46:08.372 INFO:teuthology.orchestra.run.smithi165.stdout:296 punch from 0xb9f000 to 0xba2600, (0x3600 bytes) 2024-09-17T21:46:08.384 INFO:teuthology.orchestra.run.smithi165.stdout:300 trunc from 0xe132e00 to 0x3b68a00 2024-09-17T21:46:08.421 INFO:teuthology.orchestra.run.smithi165.stdout:301 write 0xcdd0c00 thru 0xcddf56f (0xe970 bytes) 2024-09-17T21:46:08.422 INFO:teuthology.orchestra.run.smithi165.stdout:303 write 0xa3bf600 thru 0xa3cab8a (0xb58b bytes) 2024-09-17T21:46:08.431 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:08.429+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:08.437 INFO:teuthology.orchestra.run.smithi165.stdout:306 write 0xdc46400 thru 0xdc4da95 (0x7696 bytes) 2024-09-17T21:46:08.437 INFO:teuthology.orchestra.run.smithi165.stdout:307 read 0xcbfe600 thru 0xcc02f3f (0x4940 bytes) 2024-09-17T21:46:08.449 INFO:teuthology.orchestra.run.smithi165.stdout:309 trunc from 0xdc4dc00 to 0x4e4ba00 2024-09-17T21:46:08.457 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:08.457 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:08.457 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:08.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:08.476 INFO:teuthology.orchestra.run.smithi165.stdout:312 trunc from 0x4e4ba00 to 0x31d8000 2024-09-17T21:46:08.487 INFO:teuthology.orchestra.run.smithi165.stdout:315 punch from 0x1872a00 to 0x187e600, (0xbc00 bytes) 2024-09-17T21:46:08.488 INFO:teuthology.orchestra.run.smithi165.stdout:316 punch from 0x352000 to 0x358400, (0x6400 bytes) 2024-09-17T21:46:08.489 INFO:teuthology.orchestra.run.smithi165.stdout:321 read 0x245c200 thru 0x245fac2 (0x38c3 bytes) 2024-09-17T21:46:08.498 INFO:teuthology.orchestra.run.smithi165.stdout:323 trunc from 0x31d8000 to 0xb0fe200 2024-09-17T21:46:08.507 INFO:teuthology.orchestra.run.smithi165.stdout:325 read 0xa105c00 thru 0xa115719 (0xfb1a bytes) 2024-09-17T21:46:08.509 INFO:teuthology.orchestra.run.smithi165.stdout:326 read 0xa9ae400 thru 0xa9bb98b (0xd58c bytes) 2024-09-17T21:46:08.509 INFO:teuthology.orchestra.run.smithi165.stdout:328 write 0xa476400 thru 0xa47ba2f (0x5630 bytes) 2024-09-17T21:46:08.509 INFO:teuthology.orchestra.run.smithi165.stdout:332 read 0xd94a00 thru 0xd989b8 (0x3fb9 bytes) 2024-09-17T21:46:08.513 INFO:teuthology.orchestra.run.smithi165.stdout:333 read 0x5422000 thru 0x542a922 (0x8923 bytes) 2024-09-17T21:46:08.517 INFO:teuthology.orchestra.run.smithi165.stdout:336 read 0x2cfa200 thru 0x2d00bb6 (0x69b7 bytes) 2024-09-17T21:46:08.520 INFO:teuthology.orchestra.run.smithi165.stdout:339 trunc from 0xb0fe200 to 0x5d9f800 2024-09-17T21:46:08.552 INFO:teuthology.orchestra.run.smithi165.stdout:340 trunc from 0x5d9f800 to 0xdb04200 2024-09-17T21:46:08.553 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:08.551+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:08.561 INFO:teuthology.orchestra.run.smithi165.stdout:342 read 0x5602000 thru 0x561028a (0xe28b bytes) 2024-09-17T21:46:08.562 INFO:teuthology.orchestra.run.smithi165.stdout:343 read 0x678e200 thru 0x679593a (0x773b bytes) 2024-09-17T21:46:08.563 INFO:teuthology.orchestra.run.smithi165.stdout:344 clone 3 order 20 su 524288 sc 5 2024-09-17T21:46:08.623 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:08.624 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:08.624 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:08.624 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:08.624 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:08.632 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:08.633 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:08.630+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:08.733 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:08.731+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:08.733 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone2 intact 2024-09-17T21:46:08.783 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:592 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify cookie 140447443938128] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.414427973, 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022429160000000002, 2024-09-17T21:46:08.784 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 592 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365901+0000", 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9589999999999997e-06 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365908+0000", 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365914+0000", 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5010000000000004e-06 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367902+0000", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001987979 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367975+0000", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3120999999999993e-05 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368140+0000", 2024-09-17T21:46:08.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016479600000000001 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:593 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4142577670000001, 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026906790000000001, 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 593 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366068+0000", 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.063e-06 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366073+0000", 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366083+0000", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1209999999999996e-06 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366089+0000", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8449999999999997e-06 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368588+0000", 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024981970000000002 2024-09-17T21:46:08.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368678+0000", 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0329999999999997e-05 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368758+0000", 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9985000000000001e-05 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:895 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.414010454, 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790549999999998, 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 895 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366316+0000", 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1540000000000001e-06 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366321+0000", 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000004e-06 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366333+0000", 2024-09-17T21:46:08.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1968e-05 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366341+0000", 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6820000000000003e-06 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368782+0000", 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024408260000000001 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368818+0000", 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999997e-05 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368894+0000", 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5613999999999998e-05 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:896 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4137928280000001, 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024619099999999999, 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 896 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366533+0000", 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.95e-07 2024-09-17T21:46:08.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366538+0000", 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1220000000000003e-06 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366550+0000", 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1914e-05 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366558+0000", 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9719999999999992e-06 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368910+0000", 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023516679999999999 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368948+0000", 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7901999999999997e-05 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368994+0000", 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6337e-05 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:897 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.413618853, 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023804960000000002, 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 897 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366707+0000", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366713+0000", 2024-09-17T21:46:08.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366724+0000", 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1712000000000001e-05 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366732+0000", 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7489999999999998e-06 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369009+0000", 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022764590000000002 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369041+0000", 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2733e-05 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369087+0000", 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5225999999999997e-05 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:594 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.413317489, 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023664879999999999, 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 594 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367132+0000", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000124886 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367138+0000", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367149+0000", 2024-09-17T21:46:08.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.064e-05 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367156+0000", 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3649999999999998e-06 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369124+0000", 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019677969999999999 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369209+0000", 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5117999999999994e-05 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369374+0000", 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164635 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:899 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4128119720000001, 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002219216, 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 899 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367514+0000", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367517+0000", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.055e-06 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367524+0000", 2024-09-17T21:46:08.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7689999999999996e-06 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367531+0000", 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369505+0000", 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001974618 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369584+0000", 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8833999999999998e-05 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369732+0000", 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014817899999999999 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:900 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.412601963, 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021150990000000001, 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367724+0000", 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0500000000000002e-07 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367728+0000", 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9260000000000002e-06 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367738+0000", 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9010000000000006e-06 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367744+0000", 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5060000000000001e-06 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369712+0000", 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001967463 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369749+0000", 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7864000000000002e-05 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369838+0000", 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8634000000000006e-05 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:901 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4125028829999999, 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002242338, 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 901 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367823+0000", 2024-09-17T21:46:08.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367828+0000", 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6970000000000001e-06 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367838+0000", 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0077e-05 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367846+0000", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8189999999999992e-06 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369885+0000", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020395880000000002 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369966+0000", 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0611000000000005e-05 2024-09-17T21:46:08.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370064+0000", 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8665999999999994e-05 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:595 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.412344805, 2024-09-17T21:46:08.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022065299999999999, 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 595 2024-09-17T21:46:08.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:08.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367981+0000", 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367986+0000", 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1730000000000003e-06 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367996+0000", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0030999999999999e-05 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368003+0000", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1849999999999998e-06 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370083+0000", 2024-09-17T21:46:08.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020797519999999998 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370123+0000", 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9341000000000002e-05 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370187+0000", 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.41e-05 2024-09-17T21:46:08.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:596 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4122830790000001, 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023552930000000001, 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.833 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 596 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368046+0000", 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8399999999999997e-06 2024-09-17T21:46:08.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368054+0000", 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0330000000000008e-06 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368060+0000", 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9920000000000004e-06 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370213+0000", 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021526980000000002 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370290+0000", 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7007000000000003e-05 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370397+0000", 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107112 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:597 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.41225505, 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025370649999999998, 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 597 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368074+0000", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6169999999999998e-06 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368081+0000", 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2309999999999999e-06 2024-09-17T21:46:08.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368087+0000", 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3430000000000003e-06 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370416+0000", 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023292370000000001 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370491+0000", 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5036000000000005e-05 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370607+0000", 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116146 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.842 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:598 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4122297029999999, 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027463190000000001, 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 598 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368096+0000", 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368099+0000", 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0910000000000001e-06 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368106+0000", 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2010000000000002e-06 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368111+0000", 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9679999999999997e-06 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370652+0000", 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025405760000000001 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370733+0000", 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1157999999999998e-05 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370842+0000", 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000108828 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:599 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4122055630000001, 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028749090000000001, 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 599 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368120+0000", 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9800000000000004e-07 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368123+0000", 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8499999999999998e-06 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368132+0000", 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9849999999999995e-06 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368137+0000", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.31e-06 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370873+0000", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027356020000000002 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370928+0000", 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4928e-05 2024-09-17T21:46:08.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370994+0000", 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6736000000000006e-05 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:600 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.412004472, 2024-09-17T21:46:08.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028934400000000002, 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 600 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:08.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368321+0000", 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6099999999999999e-07 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368327+0000", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2000000000000002e-06 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368347+0000", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0619e-05 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368354+0000", 2024-09-17T21:46:08.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371033+0000", 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002678737 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371115+0000", 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750999999999997e-05 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371214+0000", 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8938999999999996e-05 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:902 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.411954747, 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029693020000000001, 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 902 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368372+0000", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3260000000000001e-06 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368377+0000", 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368390+0000", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2591e-05 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368400+0000", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0575e-05 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371238+0000", 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002838035 2024-09-17T21:46:08.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371280+0000", 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2001e-05 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371340+0000", 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9286000000000001e-05 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:614 3.0 3:01a50957:::rbd_object_map.1112ff4590f3.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4077601930000001, 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021228620000000001, 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 614 2024-09-17T21:46:08.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372566+0000", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.065e-06 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372572+0000", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8640000000000001e-06 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372585+0000", 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3451e-05 2024-09-17T21:46:08.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372591+0000", 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372644+0000", 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3009e-05 2024-09-17T21:46:08.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372741+0000", 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6280999999999995e-05 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373124+0000", 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038353200000000001 2024-09-17T21:46:08.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373927+0000", 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000802807 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374591+0000", 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066400900000000002 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374646+0000", 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5015000000000001e-05 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374688+0000", 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1536999999999999e-05 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:618 3.2 3:41bbdce2:::rbd_data.1112ff4590f3.0000000000000003:head [stat,truncate 4194304] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.405536906, 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00307676, 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 618 2024-09-17T21:46:08.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374789+0000", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1200000000000002e-07 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374793+0000", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0729999999999998e-06 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374806+0000", 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2927e-05 2024-09-17T21:46:08.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374814+0000", 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.712e-06 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374873+0000", 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9172000000000002e-05 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374966+0000", 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3333999999999998e-05 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.375143+0000", 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017733899999999999 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.376282+0000", 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001139064 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377810+0000", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001527668 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377840+0000", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0178000000000001e-05 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377865+0000", 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4481000000000002e-05 2024-09-17T21:46:08.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4480.0:353 3.1a e27/21 3:5bb9badb:::rbd_data.1112d1429a58.0000000000000005:head v 27'6, mlcod=26'3)", 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.46843328899999997, 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053783859999999998, 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:46:08.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314028+0000", 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021361570000000001 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314041+0000", 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2602000000000001e-05 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314046+0000", 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1880000000000001e-06 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314092+0000", 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5654e-05 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314107+0000", 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5135e-05 2024-09-17T21:46:08.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.317214+0000", 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003107126 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.317270+0000", 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5853000000000002e-05 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:395 3.0 3:010435be:::rbd_data.1112d1429a58.0000000000000006:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.44832863499999998, 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021242209999999999, 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 395 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:08.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331998+0000", 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332004+0000", 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3419999999999996e-06 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332023+0000", 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8074999999999999e-05 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332032+0000", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6600000000000007e-06 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332110+0000", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8073999999999998e-05 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332214+0000", 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103848 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332350+0000", 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013555899999999999 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332879+0000", 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052893199999999997 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334016+0000", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001137613 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334071+0000", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4391000000000002e-05 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334121+0000", 2024-09-17T21:46:08.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9907000000000002e-05 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:08.886 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:08.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:08.889 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:08.885+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:08.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:08.988 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:08.986+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:09.038 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:09.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:09.039 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:09.039 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:09.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:09.090 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:09.087+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:09.111 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:09.111 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:09.130 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:09.130 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:09.148 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:09.148 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:09.149 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-17T21:46:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.165 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.171 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:09.172 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:09.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:09.195 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:09.196 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:09.196 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:09.193+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:09.214 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:09.215 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:09.285 INFO:teuthology.orchestra.run.smithi165.stdout:348 read 0xc9aa000 thru 0xc9b824c (0xe24d bytes) 2024-09-17T21:46:09.285 INFO:teuthology.orchestra.run.smithi165.stdout:350 read 0x69bd200 thru 0x69be4db (0x12dc bytes) 2024-09-17T21:46:09.286 INFO:teuthology.orchestra.run.smithi165.stdout:351 read 0xbd1ba00 thru 0xbd219cb (0x5fcc bytes) 2024-09-17T21:46:09.287 INFO:teuthology.orchestra.run.smithi165.stdout:352 write 0x3122800 thru 0x3126dcc (0x45cd bytes) 2024-09-17T21:46:09.287 INFO:teuthology.orchestra.run.smithi165.stdout:355 punch from 0xad41800 to 0xad49400, (0x7c00 bytes) 2024-09-17T21:46:09.290 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.290 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.290 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:09.288+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:09.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:09.303 INFO:teuthology.orchestra.run.smithi165.stdout:357 trunc from 0xdb04200 to 0x9c14400 2024-09-17T21:46:09.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:46:09.322 INFO:teuthology.orchestra.run.smithi165.stdout:358 read 0x198c200 thru 0x198eb8a (0x298b bytes) 2024-09-17T21:46:09.323 INFO:teuthology.orchestra.run.smithi165.stdout:360 write 0x73c3200 thru 0x73cb320 (0x8121 bytes) 2024-09-17T21:46:09.324 INFO:teuthology.orchestra.run.smithi165.stdout:362 read 0x9291400 thru 0x92947ee (0x33ef bytes) 2024-09-17T21:46:09.341 INFO:teuthology.orchestra.run.smithi165.stdout:363 write 0xd167e00 thru 0xd169cd6 (0x1ed7 bytes) 2024-09-17T21:46:09.342 INFO:teuthology.orchestra.run.smithi165.stdout:367 write 0x1ee2600 thru 0x1eec0ac (0x9aad bytes) 2024-09-17T21:46:09.344 INFO:teuthology.orchestra.run.smithi165.stdout:368 clone 4 order 25 su 2097152 sc 10 2024-09-17T21:46:09.392 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:09.390+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:09.411 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.411 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:09.493 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:09.491+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:09.526 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.527 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:09.594 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:09.592+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:09.652 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:46:09.690 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.690 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:09.707 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:09.705+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:09.743 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone3 from 0xd169e00 (overlap 0x9c14400) to 0x5da5c00 2024-09-17T21:46:09.796 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:09.793+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:09.823 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.823 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:09.916 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:09.914+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:09.981 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:09.981 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:09.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:09.998 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:09.995+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:10.098 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:10.096+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.140 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.149 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:10.164 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-17T21:46:10.199 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:10.197+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:10.300 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:10.298+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:10.303 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.304 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:10.401 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:10.399+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:10.424 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.424 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:10.502 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:10.500+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:10.548 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.548 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:10.603 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:10.601+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:10.666 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.666 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.675 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:10.703 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:10.701+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:10.777 INFO:teuthology.orchestra.run.smithi165.stdout:369 read 0x56fda00 thru 0x57067a6 (0x8da7 bytes) 2024-09-17T21:46:10.782 INFO:teuthology.orchestra.run.smithi165.stdout:374 trunc from 0xd169e00 to 0xabda400 2024-09-17T21:46:10.782 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.782 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:10.804 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:10.802+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:10.906 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:10.903+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:10.925 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:10.926 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:10.935 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:10.986 INFO:teuthology.orchestra.run.smithi165.stdout:376 punch from 0x86ec000 to 0x86f2800, (0x6800 bytes) 2024-09-17T21:46:10.987 INFO:teuthology.orchestra.run.smithi165.stdout:377 read 0x4a87e00 thru 0x4a91c0a (0x9e0b bytes) 2024-09-17T21:46:10.989 INFO:teuthology.orchestra.run.smithi165.stdout:378 read 0x7cc4000 thru 0x7cc833c (0x433d bytes) 2024-09-17T21:46:10.989 INFO:teuthology.orchestra.run.smithi165.stdout:380 punch from 0x62f1400 to 0x62fa000, (0x8c00 bytes) 2024-09-17T21:46:10.989 INFO:teuthology.orchestra.run.smithi165.stdout:382 write 0x21f5a00 thru 0x22023de (0xc9df bytes) 2024-09-17T21:46:10.990 INFO:teuthology.orchestra.run.smithi165.stdout:383 trunc from 0xabda400 to 0xa473a00 2024-09-17T21:46:11.002 INFO:teuthology.orchestra.run.smithi165.stdout:389 read 0x3c29400 thru 0x3c32f33 (0x9b34 bytes) 2024-09-17T21:46:11.005 INFO:teuthology.orchestra.run.smithi165.stdout:392 read 0x8c41000 thru 0x8c4d2f7 (0xc2f8 bytes) 2024-09-17T21:46:11.006 INFO:teuthology.orchestra.run.smithi165.stdout:395 read 0x7e10c00 thru 0x7e1b66b (0xaa6c bytes) 2024-09-17T21:46:11.016 INFO:teuthology.orchestra.run.smithi165.stdout:396 write 0xb789200 thru 0xb793e55 (0xac56 bytes) 2024-09-17T21:46:11.019 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:11.017+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:11.021 INFO:teuthology.orchestra.run.smithi165.stdout:398 write 0x4edcc00 thru 0x4ee783c (0xac3d bytes) 2024-09-17T21:46:11.022 INFO:teuthology.orchestra.run.smithi165.stdout:399 read 0xaae3000 thru 0xaae93f9 (0x63fa bytes) 2024-09-17T21:46:11.027 INFO:teuthology.orchestra.run.smithi165.stdout:401 trunc from 0xb794000 to 0x7639c00 2024-09-17T21:46:11.046 INFO:teuthology.orchestra.run.smithi165.stdout:402 read 0x5e60c00 thru 0x5e65880 (0x4c81 bytes) 2024-09-17T21:46:11.068 INFO:teuthology.orchestra.run.smithi165.stdout:405 write 0xad23a00 thru 0xad2c48a (0x8a8b bytes) 2024-09-17T21:46:11.075 INFO:teuthology.orchestra.run.smithi165.stdout:411 read 0x6f1b200 thru 0x6f27b94 (0xc995 bytes) 2024-09-17T21:46:11.076 INFO:teuthology.orchestra.run.smithi165.stdout:412 read 0x5380a00 thru 0x5386690 (0x5c91 bytes) 2024-09-17T21:46:11.077 INFO:teuthology.orchestra.run.smithi165.stdout:415 punch from 0x3bf0000 to 0x3bfd800, (0xd800 bytes) 2024-09-17T21:46:11.089 INFO:teuthology.orchestra.run.smithi165.stdout:417 write 0xc4d3600 thru 0xc4d5525 (0x1f26 bytes) 2024-09-17T21:46:11.090 INFO:teuthology.orchestra.run.smithi165.stdout:418 read 0x4ee0a00 thru 0x4eecd2b (0xc32c bytes) 2024-09-17T21:46:11.094 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:11.094 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:11.100 INFO:teuthology.orchestra.run.smithi165.stdout:420 write 0xcbf9200 thru 0xcc02d9e (0x9b9f bytes) 2024-09-17T21:46:11.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:11.107 INFO:teuthology.orchestra.run.smithi165.stdout:421 write 0x84ac600 thru 0x84b051e (0x3f1f bytes) 2024-09-17T21:46:11.108 INFO:teuthology.orchestra.run.smithi165.stdout:426 write 0x939e000 thru 0x93a6070 (0x8071 bytes) 2024-09-17T21:46:11.108 INFO:teuthology.orchestra.run.smithi165.stdout:428 read 0x9238400 thru 0x924337b (0xaf7c bytes) 2024-09-17T21:46:11.119 INFO:teuthology.orchestra.run.smithi165.stdout:430 write 0xcebf200 thru 0xcec4945 (0x5746 bytes) 2024-09-17T21:46:11.120 INFO:teuthology.orchestra.run.smithi165.stdout:432 punch from 0xbcb0400 to 0xbcb9200, (0x8e00 bytes) 2024-09-17T21:46:11.121 INFO:teuthology.orchestra.run.smithi165.stdout:433 read 0xe6ca00 thru 0xe71ac0 (0x50c1 bytes) 2024-09-17T21:46:11.121 INFO:teuthology.orchestra.run.smithi165.stdout:435 read 0xcaeb800 thru 0xcaf2a4c (0x724d bytes) 2024-09-17T21:46:11.122 INFO:teuthology.orchestra.run.smithi165.stdout:439 write 0x9fc1a00 thru 0x9fc9c3b (0x823c bytes) 2024-09-17T21:46:11.122 INFO:teuthology.orchestra.run.smithi165.stdout:440 read 0x52a5e00 thru 0x52af847 (0x9a48 bytes) 2024-09-17T21:46:11.123 INFO:teuthology.orchestra.run.smithi165.stdout:441 punch from 0x5860600 to 0x5860e00, (0x800 bytes) 2024-09-17T21:46:11.123 INFO:teuthology.orchestra.run.smithi165.stdout:444 read 0x406b600 thru 0x40746fd (0x90fe bytes) 2024-09-17T21:46:11.128 INFO:teuthology.orchestra.run.smithi165.stdout:445 trunc from 0xcec4a00 to 0x976ea00 2024-09-17T21:46:11.140 INFO:teuthology.orchestra.run.smithi165.stdout:446 write 0x1452a00 thru 0x145e0b9 (0xb6ba bytes) 2024-09-17T21:46:11.142 INFO:teuthology.orchestra.run.smithi165.stdout:447 write 0x7d5b600 thru 0x7d67a20 (0xc421 bytes) 2024-09-17T21:46:11.142 INFO:teuthology.orchestra.run.smithi165.stdout:448 read 0x746aa00 thru 0x746ff01 (0x5502 bytes) 2024-09-17T21:46:11.142 INFO:teuthology.orchestra.run.smithi165.stdout:450 punch from 0x162b400 to 0x162ea00, (0x3600 bytes) 2024-09-17T21:46:11.143 INFO:teuthology.orchestra.run.smithi165.stdout:451 read 0x6552600 thru 0x6562421 (0xfe22 bytes) 2024-09-17T21:46:11.145 INFO:teuthology.orchestra.run.smithi165.stdout:452 write 0x76d4800 thru 0x76d941f (0x4c20 bytes) 2024-09-17T21:46:11.145 INFO:teuthology.orchestra.run.smithi165.stdout:455 read 0x208d800 thru 0x20967c3 (0x8fc4 bytes) 2024-09-17T21:46:11.145 INFO:teuthology.orchestra.run.smithi165.stdout:460 read 0x750f000 thru 0x7519887 (0xa888 bytes) 2024-09-17T21:46:11.146 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:11.144+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:11.146 INFO:teuthology.orchestra.run.smithi165.stdout:461 read 0x2523200 thru 0x2532674 (0xf475 bytes) 2024-09-17T21:46:11.147 INFO:teuthology.orchestra.run.smithi165.stdout:462 read 0x7397c00 thru 0x739b2f1 (0x36f2 bytes) 2024-09-17T21:46:11.147 INFO:teuthology.orchestra.run.smithi165.stdout:463 read 0xecc800 thru 0xed4e3e (0x863f bytes) 2024-09-17T21:46:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:464 write 0xe9e6000 thru 0xe9efac8 (0x9ac9 bytes) 2024-09-17T21:46:11.165 INFO:teuthology.orchestra.run.smithi165.stdout:465 write 0x214ca00 thru 0x21524fb (0x5afc bytes) 2024-09-17T21:46:11.165 INFO:teuthology.orchestra.run.smithi165.stdout:466 write 0x79d8200 thru 0x79e18da (0x96db bytes) 2024-09-17T21:46:11.166 INFO:teuthology.orchestra.run.smithi165.stdout:468 read 0x5a05800 thru 0x5a07dc6 (0x25c7 bytes) 2024-09-17T21:46:11.167 INFO:teuthology.orchestra.run.smithi165.stdout:469 read 0xaae2400 thru 0xaae612e (0x3d2f bytes) 2024-09-17T21:46:11.167 INFO:teuthology.orchestra.run.smithi165.stdout:471 read 0xbb3e800 thru 0xbb48c73 (0xa474 bytes) 2024-09-17T21:46:11.168 INFO:teuthology.orchestra.run.smithi165.stdout:472 write 0x5f5e600 thru 0x5f66940 (0x8341 bytes) 2024-09-17T21:46:11.168 INFO:teuthology.orchestra.run.smithi165.stdout:474 write 0x75ec400 thru 0x75f3551 (0x7152 bytes) 2024-09-17T21:46:11.169 INFO:teuthology.orchestra.run.smithi165.stdout:475 read 0xaedee00 thru 0xaee0c44 (0x1e45 bytes) 2024-09-17T21:46:11.173 INFO:teuthology.orchestra.run.smithi165.stdout:479 read 0xb547200 thru 0xb5561c0 (0xefc1 bytes) 2024-09-17T21:46:11.173 INFO:teuthology.orchestra.run.smithi165.stdout:481 clone 5 order 19 su 4096 sc 12 2024-09-17T21:46:11.207 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:11.204+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:11.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:11.259 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:11.260 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:11.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:11.309 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:11.307+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:11.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:11.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:11.371 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:11.371 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:11.387 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:11.388 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:11.405 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:11.409 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:11.409 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:11.409 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:11.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-17T21:46:11.410 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:11.407+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:11.420 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:592 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify cookie 140447443938128] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0516697820000003, 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022429160000000002, 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 592 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365897+0000", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365901+0000", 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9589999999999997e-06 2024-09-17T21:46:11.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365908+0000", 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0389999999999999e-06 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365914+0000", 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5010000000000004e-06 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367902+0000", 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001987979 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367975+0000", 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3120999999999993e-05 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368140+0000", 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016479600000000001 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:593 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0514995760000003, 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026906790000000001, 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 593 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366067+0000", 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366068+0000", 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.063e-06 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366073+0000", 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1379999999999998e-06 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366083+0000", 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1209999999999996e-06 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366089+0000", 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8449999999999997e-06 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368588+0000", 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024981970000000002 2024-09-17T21:46:11.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368678+0000", 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0329999999999997e-05 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368758+0000", 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9985000000000001e-05 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:895 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0512522630000003, 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025790549999999998, 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 895 2024-09-17T21:46:11.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366314+0000", 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366316+0000", 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1540000000000001e-06 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366321+0000", 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4110000000000004e-06 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366333+0000", 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1968e-05 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366341+0000", 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6820000000000003e-06 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368782+0000", 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024408260000000001 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368818+0000", 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6399999999999997e-05 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368894+0000", 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5613999999999998e-05 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:896 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0510346369999999, 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024619099999999999, 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 896 2024-09-17T21:46:11.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366532+0000", 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366533+0000", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.95e-07 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366538+0000", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1220000000000003e-06 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366550+0000", 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1914e-05 2024-09-17T21:46:11.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366558+0000", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9719999999999992e-06 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368910+0000", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023516679999999999 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368948+0000", 2024-09-17T21:46:11.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7901999999999997e-05 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368994+0000", 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6337e-05 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:897 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0508606619999998, 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023804960000000002, 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 897 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366706+0000", 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366707+0000", 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3600000000000002e-07 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366713+0000", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366724+0000", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1712000000000001e-05 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366732+0000", 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7489999999999998e-06 2024-09-17T21:46:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369009+0000", 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022764590000000002 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369041+0000", 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2733e-05 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369087+0000", 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5225999999999997e-05 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:594 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0505592979999996, 2024-09-17T21:46:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023664879999999999, 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 594 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367007+0000", 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367132+0000", 2024-09-17T21:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000124886 2024-09-17T21:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367138+0000", 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367149+0000", 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.064e-05 2024-09-17T21:46:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367156+0000", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3649999999999998e-06 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369124+0000", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019677969999999999 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369209+0000", 2024-09-17T21:46:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5117999999999994e-05 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369374+0000", 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000164635 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:899 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0500537809999999, 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002219216, 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 899 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367513+0000", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367514+0000", 2024-09-17T21:46:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367517+0000", 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.055e-06 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367524+0000", 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7689999999999996e-06 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367531+0000", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1210000000000003e-06 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369505+0000", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001974618 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369584+0000", 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8833999999999998e-05 2024-09-17T21:46:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369732+0000", 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014817899999999999 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:900 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.049843772, 2024-09-17T21:46:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021150990000000001, 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367723+0000", 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367724+0000", 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0500000000000002e-07 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367728+0000", 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9260000000000002e-06 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367738+0000", 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9010000000000006e-06 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367744+0000", 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5060000000000001e-06 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369712+0000", 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001967463 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369749+0000", 2024-09-17T21:46:11.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7864000000000002e-05 2024-09-17T21:46:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369838+0000", 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8634000000000006e-05 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:901 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.049744692, 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002242338, 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 901 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367822+0000", 2024-09-17T21:46:11.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367823+0000", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8000000000000004e-07 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367828+0000", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6970000000000001e-06 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367838+0000", 2024-09-17T21:46:11.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0077e-05 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367846+0000", 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8189999999999992e-06 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369885+0000", 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020395880000000002 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369966+0000", 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0611000000000005e-05 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370064+0000", 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8665999999999994e-05 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:595 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0495866139999999, 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022065299999999999, 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 595 2024-09-17T21:46:11.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367980+0000", 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367981+0000", 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4799999999999997e-07 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367986+0000", 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1730000000000003e-06 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367996+0000", 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0030999999999999e-05 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368003+0000", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1849999999999998e-06 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370083+0000", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020797519999999998 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370123+0000", 2024-09-17T21:46:11.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9341000000000002e-05 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370187+0000", 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.41e-05 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:596 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0495248879999997, 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023552930000000001, 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 596 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368042+0000", 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1099999999999995e-07 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368046+0000", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8399999999999997e-06 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368054+0000", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0330000000000008e-06 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368060+0000", 2024-09-17T21:46:11.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9920000000000004e-06 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370213+0000", 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021526980000000002 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370290+0000", 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7007000000000003e-05 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370397+0000", 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000107112 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:597 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0494968589999996, 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025370649999999998, 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 597 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368070+0000", 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368074+0000", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6169999999999998e-06 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368081+0000", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2309999999999999e-06 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368087+0000", 2024-09-17T21:46:11.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3430000000000003e-06 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370416+0000", 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023292370000000001 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370491+0000", 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5036000000000005e-05 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370607+0000", 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116146 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:598 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0494715120000002, 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027463190000000001, 2024-09-17T21:46:11.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 598 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368095+0000", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368096+0000", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368099+0000", 2024-09-17T21:46:11.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0910000000000001e-06 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368106+0000", 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2010000000000002e-06 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368111+0000", 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9679999999999997e-06 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370652+0000", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025405760000000001 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370733+0000", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1157999999999998e-05 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370842+0000", 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000108828 2024-09-17T21:46:11.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:599 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0494473720000004, 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028749090000000001, 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 599 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368119+0000", 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368120+0000", 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9800000000000004e-07 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368123+0000", 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8499999999999998e-06 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368132+0000", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9849999999999995e-06 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368137+0000", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.31e-06 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370873+0000", 2024-09-17T21:46:11.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027356020000000002 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370928+0000", 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4928e-05 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.370994+0000", 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6736000000000006e-05 2024-09-17T21:46:11.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:600 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0492462810000003, 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028934400000000002, 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 600 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368320+0000", 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368321+0000", 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6099999999999999e-07 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368327+0000", 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2000000000000002e-06 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368347+0000", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0619e-05 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368354+0000", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371033+0000", 2024-09-17T21:46:11.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002678737 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371115+0000", 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1750999999999997e-05 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371214+0000", 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8938999999999996e-05 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:902 3.0 3:02fc9c8c:::rbd_header.1112ff4590f3:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0491965560000001, 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029693020000000001, 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 902 2024-09-17T21:46:11.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368370+0000", 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368372+0000", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3260000000000001e-06 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368377+0000", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4879999999999998e-06 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368390+0000", 2024-09-17T21:46:11.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2591e-05 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368400+0000", 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0575e-05 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371238+0000", 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002838035 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371280+0000", 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2001e-05 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.371340+0000", 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9286000000000001e-05 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:614 3.0 3:01a50957:::rbd_object_map.1112ff4590f3.0000000000000004:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0450020020000004, 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021228620000000001, 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 614 2024-09-17T21:46:11.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372565+0000", 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372566+0000", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.065e-06 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372572+0000", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8640000000000001e-06 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372585+0000", 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3451e-05 2024-09-17T21:46:11.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372591+0000", 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2920000000000001e-06 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372644+0000", 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3009e-05 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.372741+0000", 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6280999999999995e-05 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373124+0000", 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038353200000000001 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373927+0000", 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000802807 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374591+0000", 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066400900000000002 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:11.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374646+0000", 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5015000000000001e-05 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374688+0000", 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1536999999999999e-05 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:618 3.2 3:41bbdce2:::rbd_data.1112ff4590f3.0000000000000003:head [stat,truncate 4194304] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0427787149999999, 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00307676, 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 618 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374788+0000", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374789+0000", 2024-09-17T21:46:11.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1200000000000002e-07 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374793+0000", 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0729999999999998e-06 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374806+0000", 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2927e-05 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374814+0000", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.712e-06 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374873+0000", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9172000000000002e-05 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374966+0000", 2024-09-17T21:46:11.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3333999999999998e-05 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 5", 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.375143+0000", 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017733899999999999 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.376282+0000", 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001139064 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377810+0000", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001527668 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377840+0000", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0178000000000001e-05 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.377865+0000", 2024-09-17T21:46:11.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4481000000000002e-05 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4480.0:353 3.1a e27/21 3:5bb9badb:::rbd_data.1112d1429a58.0000000000000005:head v 27'6, mlcod=26'3)", 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.1056750979999999, 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053783859999999998, 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311892+0000", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314028+0000", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021361570000000001 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314041+0000", 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2602000000000001e-05 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314046+0000", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1880000000000001e-06 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314092+0000", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5654e-05 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.314107+0000", 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5135e-05 2024-09-17T21:46:11.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.317214+0000", 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003107126 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.317270+0000", 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5853000000000002e-05 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:395 3.0 3:010435be:::rbd_data.1112d1429a58.0000000000000006:head [stat,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.085570444, 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021242209999999999, 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 395 2024-09-17T21:46:11.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331996+0000", 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331998+0000", 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8199999999999999e-06 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332004+0000", 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3419999999999996e-06 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332023+0000", 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8074999999999999e-05 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332032+0000", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6600000000000007e-06 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332110+0000", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8073999999999998e-05 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332214+0000", 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103848 2024-09-17T21:46:11.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332350+0000", 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00013555899999999999 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332879+0000", 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052893199999999997 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:11.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334016+0000", 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001137613 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334071+0000", 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4391000000000002e-05 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334121+0000", 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9907000000000002e-05 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:11.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:11.527 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:11.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:11.528 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:11.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:11.529 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:11.530 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:11.530 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:11.526+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:11.629 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:11.627+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:11.667 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:46:11.682 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:11.683 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:11.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:11.728 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:11.726+0000 7f0c28614640 -1 mon.a@0(leader).osd e30 definitely_dead 0 2024-09-17T21:46:11.730 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:11.728+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:11.760 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone4 from 0xe9efc00 (overlap 0x7639c00) to 0x6a67200 2024-09-17T21:46:11.831 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:11.829+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:11.845 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:11.846 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:11.846 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:11.846 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:11.846 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:11.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:11.939 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:11.937+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:12.010 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:12.010 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:12.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:12.010 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:578 3.7 3:e0ee91cb:::rbd_data.1112ff4590f3.0000000000000008:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365646+0000", 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6410346909999998, 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030439360000000001, 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 578 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365646+0000", 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365646+0000", 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365647+0000", 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8800000000000002e-07 2024-09-17T21:46:12.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365651+0000", 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5759999999999999e-06 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365669+0000", 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8076000000000001e-05 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365675+0000", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365755+0000", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9614e-05 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365850+0000", 2024-09-17T21:46:12.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5189e-05 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366049+0000", 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019900399999999999 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367329+0000", 2024-09-17T21:46:12.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012800839999999999 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368584+0000", 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001254682 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368645+0000", 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0847999999999997e-05 2024-09-17T21:46:12.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368690+0000", 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5354e-05 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:584 3.8 3:10bbbd7b:::rbd_data.1112ff4590f3.000000000000000e:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365712+0000", 2024-09-17T21:46:12.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6409689040000002, 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038404229999999999, 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 584 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365712+0000", 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365712+0000", 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365712+0000", 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365720+0000", 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4529999999999996e-06 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365726+0000", 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.241e-06 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365730+0000", 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0080000000000004e-06 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365787+0000", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6678999999999997e-05 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365871+0000", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4395999999999996e-05 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366045+0000", 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000174461 2024-09-17T21:46:12.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368221+0000", 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021759180000000002 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369478+0000", 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001256627 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369523+0000", 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5170999999999998e-05 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369552+0000", 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8989999999999999e-05 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:586 3.1e 3:7c547442:::rbd_data.1112ff4590f3.000000000000000f:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365754+0000", 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6409265279999996, 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024804580000000001, 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 586 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365754+0000", 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365754+0000", 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365755+0000", 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1500000000000004e-07 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0359999999999998e-06 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365766+0000", 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8120000000000008e-06 2024-09-17T21:46:12.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365772+0000", 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365838+0000", 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5852999999999994e-05 2024-09-17T21:46:12.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365922+0000", 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3978000000000001e-05 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.366113+0000", 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000191132 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367811+0000", 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016982080000000001 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368138+0000", 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00032702000000000002 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368190+0000", 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1684000000000002e-05 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368235+0000", 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5021999999999997e-05 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:589 3.8 3:14076d16:::rbd_data.1112ff4590f3.0000000000000018:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.365799+0000", 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6408817139999998, 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004071252, 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:12.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 589 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365799+0000", 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365799+0000", 2024-09-17T21:46:12.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365800+0000", 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365811+0000", 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1904e-05 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365820+0000", 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8550000000000008e-06 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.365826+0000", 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0079999999999999e-06 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367210+0000", 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013839900000000001 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367296+0000", 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5488999999999994e-05 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.367450+0000", 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015416400000000001 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.368564+0000", 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011140220000000001 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369811+0000", 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001246624 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369843+0000", 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2113999999999998e-05 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.369870+0000", 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7484e-05 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4465.0:616 3.1e 3:79f8efe5:::rbd_data.1112ff4590f3.0000000000000000:head [stat,truncate 6291456] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:07.373719+0000", 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6329613810000003, 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023613570000000001, 2024-09-17T21:46:12.041 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4465", 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 616 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373719+0000", 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373719+0000", 2024-09-17T21:46:12.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373721+0000", 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2410000000000001e-06 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373726+0000", 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3419999999999998e-06 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373739+0000", 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3227999999999999e-05 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373747+0000", 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4719999999999998e-06 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373799+0000", 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2006000000000003e-05 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.373873+0000", 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4278999999999998e-05 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:12.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.374085+0000", 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021178599999999999 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.375880+0000", 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017949070000000001 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.376021+0000", 2024-09-17T21:46:12.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014159 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.376056+0000", 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4406000000000002e-05 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:07.376081+0000", 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.51e-05 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:298 3.1e 3:79f8efe5:::rbd_data.1112ff4590f3.0000000000000000:4 [sparse-read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300111+0000", 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7065698199999999, 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020652969999999998, 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 298 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300111+0000", 2024-09-17T21:46:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300111+0000", 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300112+0000", 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6700000000000003e-07 2024-09-17T21:46:12.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300115+0000", 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9809999999999999e-06 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300122+0000", 2024-09-17T21:46:12.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5329999999999997e-06 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300128+0000", 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302051+0000", 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019232299999999999 2024-09-17T21:46:12.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302116+0000", 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5621000000000001e-05 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302176+0000", 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9775000000000003e-05 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.054 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:299 3.1e 3:7e9e5932:::rbd_data.1112ff4590f3.0000000000000002:4 [sparse-read 1048576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300136+0000", 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7065449340000001, 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022595670000000001, 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.055 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 299 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300136+0000", 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300136+0000", 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300136+0000", 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300138+0000", 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2369999999999999e-06 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300145+0000", 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0559999999999999e-06 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300150+0000", 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-06 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302215+0000", 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020649570000000001 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302299+0000", 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3419999999999998e-05 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302395+0000", 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6688000000000007e-05 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:307 3.1e 3:7c547442:::rbd_data.1112ff4590f3.000000000000000f:4 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300214+0000", 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7064666490000002, 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023361129999999999, 2024-09-17T21:46:12.060 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 307 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300214+0000", 2024-09-17T21:46:12.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300214+0000", 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300214+0000", 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3700000000000001e-07 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300217+0000", 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1730000000000002e-06 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300227+0000", 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0898e-05 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300234+0000", 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302432+0000", 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0021982759999999999 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302487+0000", 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4542000000000001e-05 2024-09-17T21:46:12.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302550+0000", 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3375999999999998e-05 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:309 3.1e 3:79f8efe5:::rbd_data.1112ff4590f3.0000000000000000:4 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300351+0000", 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7063300429999999, 2024-09-17T21:46:12.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024409729999999999, 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 309 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300351+0000", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300351+0000", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300352+0000", 2024-09-17T21:46:12.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7000000000000003e-07 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300356+0000", 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3549999999999998e-06 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300366+0000", 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0149999999999999e-05 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300373+0000", 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7630000000000003e-06 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302586+0000", 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022128669999999999 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302676+0000", 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9847999999999998e-05 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302792+0000", 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011602 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:310 3.1e 3:7e9e5932:::rbd_data.1112ff4590f3.0000000000000002:4 [sparse-read 2097152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300384+0000", 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7062965829999999, 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025402440000000001, 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 310 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300384+0000", 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300384+0000", 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300385+0000", 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1500000000000005e-07 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300387+0000", 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6410000000000002e-06 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300394+0000", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300399+0000", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3750000000000002e-06 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302825+0000", 2024-09-17T21:46:12.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024256400000000002 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302868+0000", 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3228e-05 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302924+0000", 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6060999999999997e-05 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:316 3.1e 3:7c547442:::rbd_data.1112ff4590f3.000000000000000f:4 [sparse-read 4194304~39424] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300458+0000", 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7062227449999998, 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028151069999999999, 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 316 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300458+0000", 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300458+0000", 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300458+0000", 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300462+0000", 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0080000000000004e-06 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300469+0000", 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300474+0000", 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1390000000000003e-06 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.302961+0000", 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024874060000000002 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303048+0000", 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6514000000000006e-05 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303273+0000", 2024-09-17T21:46:12.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022529200000000001 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:319 3.1e 3:79f8efe5:::rbd_data.1112ff4590f3.0000000000000000:4 [sparse-read 7340032~45568] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300481+0000", 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7061999869999998, 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031202339999999999, 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 319 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300481+0000", 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300481+0000", 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300481+0000", 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.63e-07 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300483+0000", 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300489+0000", 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0589999999999999e-06 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300494+0000", 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8380000000000001e-06 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303317+0000", 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0028226010000000001 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303411+0000", 2024-09-17T21:46:12.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3795000000000006e-05 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303601+0000", 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019023799999999999 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:320 3.1e 3:7e9e5932:::rbd_data.1112ff4590f3.0000000000000002:4 [sparse-read 3145728~56832] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300501+0000", 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.706179938, 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034163930000000002, 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 320 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300501+0000", 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300501+0000", 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300501+0000", 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300503+0000", 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-06 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300509+0000", 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1190000000000002e-06 2024-09-17T21:46:12.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300521+0000", 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1469e-05 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303631+0000", 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031101340000000001 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303680+0000", 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9064999999999999e-05 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303917+0000", 2024-09-17T21:46:12.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023714499999999999 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:330 3.1e 3:7c547442:::rbd_data.1112ff4590f3.000000000000000f:4 [sparse-read 5242880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.300609+0000", 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7060714020000001, 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034870420000000001, 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 330 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300609+0000", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300609+0000", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300610+0000", 2024-09-17T21:46:12.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300612+0000", 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-06 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300618+0000", 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0789999999999997e-06 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.300623+0000", 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9370000000000003e-06 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.303939+0000", 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033159629999999999 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304014+0000", 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5018000000000006e-05 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304096+0000", 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2398999999999995e-05 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:390 3.18 3:191761b3:::rbd_data.1112d1429a58.0000000000000001:head [stat,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.331899+0000", 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.6747817989999998, 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024655850000000002, 2024-09-17T21:46:12.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331899+0000", 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331899+0000", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331900+0000", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2600000000000001e-07 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331903+0000", 2024-09-17T21:46:12.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5300000000000001e-06 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331911+0000", 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6109999999999996e-06 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331926+0000", 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5648999999999999e-05 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332218+0000", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029176000000000001 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332297+0000", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8781000000000003e-05 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332432+0000", 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135 2024-09-17T21:46:12.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.333531+0000", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010986959999999999 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334259+0000", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072836400000000001 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334318+0000", 2024-09-17T21:46:12.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9106e-05 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334364+0000", 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6261999999999999e-05 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:391 3.18 3:1e9b0b8c:::rbd_data.1112d1429a58.0000000000000002:head [stat,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.331937+0000", 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.674743522, 2024-09-17T21:46:12.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025211230000000001, 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331937+0000", 2024-09-17T21:46:12.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331937+0000", 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331937+0000", 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331945+0000", 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2529999999999998e-06 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331952+0000", 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2119999999999997e-06 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.331958+0000", 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5799999999999999e-06 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332782+0000", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00082457899999999998 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.332877+0000", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4667999999999996e-05 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.333011+0000", 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000133943 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.333886+0000", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00087549199999999996 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334403+0000", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000516319 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334434+0000", 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1236999999999999e-05 2024-09-17T21:46:12.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.334458+0000", 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4505999999999999e-05 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:746 3.18 3:191761b3:::rbd_data.1112d1429a58.0000000000000001:head [stat,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.477038+0000", 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5296426159999998, 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002289634, 2024-09-17T21:46:12.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 746 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477038+0000", 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477038+0000", 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477039+0000", 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3500000000000005e-07 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477043+0000", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477054+0000", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0811e-05 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477061+0000", 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5410000000000003e-06 2024-09-17T21:46:12.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477101+0000", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9125999999999998e-05 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477150+0000", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9270000000000001e-05 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477275+0000", 2024-09-17T21:46:12.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012521 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.478078+0000", 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080299600000000005 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479272+0000", 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011934669999999999 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479304+0000", 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2818999999999999e-05 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479328+0000", 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3346999999999999e-05 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:747 3.18 3:1e9b0b8c:::rbd_data.1112d1429a58.0000000000000002:head [stat,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.477076+0000", 2024-09-17T21:46:12.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5296044520000001, 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00237272, 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 747 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477076+0000", 2024-09-17T21:46:12.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477076+0000", 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477077+0000", 2024-09-17T21:46:12.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3200000000000005e-07 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477080+0000", 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0199999999999999e-06 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477087+0000", 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0990000000000001e-06 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477093+0000", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-06 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477672+0000", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057962499999999995 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477753+0000", 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0428999999999999e-05 2024-09-17T21:46:12.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.477880+0000", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012754899999999999 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.478462+0000", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058164700000000002 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479373+0000", 2024-09-17T21:46:12.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00091161699999999998 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479407+0000", 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3251999999999999e-05 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.479449+0000", 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2200000000000003e-05 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4542.0:1218 3.9 e30/21 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head v 30'67, mlcod=29'64)", 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.23152526000000001, 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067374460000000002, 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:12.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775156+0000", 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775164+0000", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775185+0000", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0710999999999999e-05 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775197+0000", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1739000000000001e-05 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775269+0000", 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775290+0000", 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0866999999999999e-05 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781826+0000", 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065362390000000001 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781893+0000", 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6446999999999996e-05 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.196756497, 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:12.126 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:12.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:12.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:12.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:12.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:12.134 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:12.131+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:12.234 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:12.231+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:12.244 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:12.244 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:12.244 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:12.244 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:12.253 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:12.266 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-17T21:46:12.335 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:12.333+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:12.409 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:12.409 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:12.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:12.435 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:12.433+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:12.536 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:12.534+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:12.538 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:12.538 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:12.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.375980127999998, 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:12.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:12.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:12.546 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.253456513, 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051382629999999997, 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 37 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:12.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281325+0000", 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281326+0000", 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7999999999999995e-07 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281330+0000", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281341+0000", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0665e-05 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.281347+0000", 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7189999999999998e-06 2024-09-17T21:46:12.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285455+0000", 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041082610000000002 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285510+0000", 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4737000000000001e-05 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.285592+0000", 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2515000000000005e-05 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286064+0000", 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047146999999999998 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286389+0000", 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032487100000000001 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286422+0000", 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2838e-05 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.286463+0000", 2024-09-17T21:46:12.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1902e-05 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.064161702, 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055199849999999998, 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 113 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:12.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470620+0000", 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470621+0000", 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7000000000000004e-07 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470626+0000", 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0490000000000003e-06 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470633+0000", 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.413e-06 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470640+0000", 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474317+0000", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036775190000000002 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474366+0000", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8820999999999999e-05 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474443+0000", 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6656999999999995e-05 2024-09-17T21:46:12.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475503+0000", 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010601300000000001 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476084+0000", 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058080899999999995 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476116+0000", 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2240999999999998e-05 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476140+0000", 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4138000000000002e-05 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.064131757999998, 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059845510000000003, 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 114 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:12.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470650+0000", 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1500000000000005e-07 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470654+0000", 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:46:12.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470661+0000", 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470666+0000", 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.474985+0000", 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0043186580000000004 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475039+0000", 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3964e-05 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475133+0000", 2024-09-17T21:46:12.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4306000000000001e-05 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476574+0000", 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014412190000000001 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476585+0000", 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0852999999999999e-05 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476612+0000", 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7481e-05 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476634+0000", 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1963999999999998e-05 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.064105143999999, 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064121400000000002, 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 115 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.568 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470677+0000", 2024-09-17T21:46:12.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1500000000000005e-07 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470737+0000", 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9543000000000002e-05 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470760+0000", 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2949999999999999e-05 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470766+0000", 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4699999999999999e-06 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475535+0000", 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047688649999999997 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475586+0000", 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1138999999999999e-05 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.475664+0000", 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7914e-05 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476984+0000", 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013201210000000001 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477027+0000", 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3154000000000001e-05 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477059+0000", 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1593000000000001e-05 2024-09-17T21:46:12.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477089+0000", 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9876e-05 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:12.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.064000895, 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0064079999999999996, 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 116 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470781+0000", 2024-09-17T21:46:12.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3799999999999997e-07 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470786+0000", 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470797+0000", 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0589e-05 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470802+0000", 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8250000000000001e-06 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476175+0000", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053730119999999999 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476221+0000", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5456e-05 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476298+0000", 2024-09-17T21:46:12.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6938000000000001e-05 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477120+0000", 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082225899999999999 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477129+0000", 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3950000000000007e-06 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477163+0000", 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3238000000000003e-05 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477189+0000", 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6077000000000001e-05 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.063965966000001, 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0066701130000000001, 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.582 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 117 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470816+0000", 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3399999999999999e-07 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470822+0000", 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3900000000000001e-06 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470830+0000", 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4910000000000005e-06 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.470835+0000", 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1540000000000001e-06 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476659+0000", 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0058232529999999996 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476707+0000", 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8656000000000003e-05 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.476785+0000", 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8014000000000002e-05 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477231+0000", 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044546899999999999 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477434+0000", 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000203477 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477464+0000", 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0261999999999999e-05 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.477486+0000", 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1413e-05 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:12.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716533164000001, 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051643319999999998, 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 134 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.589 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818248+0000", 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818249+0000", 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818295+0000", 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6e-05 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818301+0000", 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6620000000000002e-06 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818322+0000", 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0877999999999999e-05 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.820797+0000", 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024750240000000001 2024-09-17T21:46:12.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.820881+0000", 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4895999999999995e-05 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821021+0000", 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139209 2024-09-17T21:46:12.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823308+0000", 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002287182 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823322+0000", 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4503e-05 2024-09-17T21:46:12.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823373+0000", 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0670000000000001e-05 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823413+0000", 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9798000000000001e-05 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716453203, 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052327600000000004, 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 135 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818328+0000", 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818329+0000", 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818332+0000", 2024-09-17T21:46:12.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7760000000000002e-06 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818337+0000", 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8050000000000003e-06 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818342+0000", 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821563+0000", 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032210400000000001 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821617+0000", 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3523000000000002e-05 2024-09-17T21:46:12.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.821691+0000", 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3685999999999999e-05 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823453+0000", 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017626619999999999 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823506+0000", 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2828999999999999e-05 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823532+0000", 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5525999999999999e-05 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823561+0000", 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9595e-05 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716430823, 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052620200000000001, 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 136 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818351+0000", 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818355+0000", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.997e-06 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818363+0000", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9079999999999995e-06 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818366+0000", 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:12.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822144+0000", 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037780029999999998 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822207+0000", 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2237999999999996e-05 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822283+0000", 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6884000000000005e-05 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823499+0000", 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012151550000000001 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823567+0000", 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8522999999999994e-05 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823590+0000", 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2340999999999998e-05 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823613+0000", 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3346999999999999e-05 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716406631000002, 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069329190000000001, 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 137 2024-09-17T21:46:12.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818375+0000", 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-07 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818444+0000", 2024-09-17T21:46:12.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8785e-05 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818456+0000", 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1542000000000001e-05 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818460+0000", 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0589999999999996e-06 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822570+0000", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041107909999999999 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822653+0000", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2986999999999997e-05 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.822792+0000", 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013813900000000001 2024-09-17T21:46:12.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825200+0000", 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024085249999999999 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825216+0000", 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6308e-05 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825266+0000", 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9221000000000002e-05 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825308+0000", 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2262999999999998e-05 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716314045000001, 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0069445879999999998, 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 138 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818468+0000", 2024-09-17T21:46:12.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8000000000000001e-07 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818471+0000", 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7429999999999998e-06 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818477+0000", 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0159999999999997e-06 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818480+0000", 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.818e-06 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823651+0000", 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005171503 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823703+0000", 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1433e-05 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.823778+0000", 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5663999999999999e-05 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825316+0000", 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001537973 2024-09-17T21:46:12.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825350+0000", 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3673000000000001e-05 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825389+0000", 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8983000000000003e-05 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825412+0000", 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3402000000000001e-05 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716297947000001, 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0071013059999999999, 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 139 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818484+0000", 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818488+0000", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7400000000000002e-06 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818492+0000", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2660000000000003e-06 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818495+0000", 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.909e-06 2024-09-17T21:46:12.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824054+0000", 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055592949999999997 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824099+0000", 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4974e-05 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824185+0000", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825469+0000", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001283246 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825482+0000", 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825532+0000", 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9475000000000003e-05 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825585+0000", 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3074e-05 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716279909000001, 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00759708, 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 140 2024-09-17T21:46:12.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818502+0000", 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8299999999999998e-07 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818554+0000", 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2130999999999999e-05 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818559+0000", 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7360000000000001e-06 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818562+0000", 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.932e-06 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824488+0000", 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059259680000000002 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824572+0000", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4559000000000001e-05 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.824692+0000", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011944599999999999 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826018+0000", 2024-09-17T21:46:12.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013263439999999999 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826046+0000", 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7908000000000001e-05 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826076+0000", 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9513999999999999e-05 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826099+0000", 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3258999999999999e-05 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.716213545999999, 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0080377260000000002, 2024-09-17T21:46:12.636 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 141 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818568+0000", 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818569+0000", 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818572+0000", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6710000000000001e-06 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818577+0000", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4700000000000004e-06 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.818581+0000", 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6020000000000002e-06 2024-09-17T21:46:12.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825609+0000", 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0070274500000000002 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825656+0000", 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7697000000000001e-05 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.825774+0000", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000117489 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826297+0000", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052359800000000001 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826505+0000", 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020754000000000001 2024-09-17T21:46:12.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826558+0000", 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3292999999999999e-05 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.826606+0000", 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7500999999999998e-05 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4480.0:221 3.1f e27/21 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head v 27'12, mlcod=25'10)", 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.286449+0000", 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2483331089999998, 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0056695340000000004, 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.286449+0000", 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.286449+0000", 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.286449+0000", 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4799999999999998e-07 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.288763+0000", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023142309999999999 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.288779+0000", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5934999999999999e-05 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.288788+0000", 2024-09-17T21:46:12.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3149999999999992e-06 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.288843+0000", 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5319999999999999e-05 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.288863+0000", 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9675000000000001e-05 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.292081+0000", 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003218463 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.292118+0000", 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7047e-05 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:356 3.14 3:2a14d8ba:::rbd_data.1112d1429a58.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:12.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.309825+0000", 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2249564749999999, 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067887470000000004, 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 356 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309825+0000", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309825+0000", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309826+0000", 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1489999999999999e-06 2024-09-17T21:46:12.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.312580+0000", 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027532590000000001 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.312601+0000", 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1073e-05 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.312611+0000", 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8919999999999995e-06 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.312679+0000", 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8491999999999994e-05 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.312799+0000", 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000119743 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313028+0000", 2024-09-17T21:46:12.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000229168 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316091+0000", 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030629749999999999 2024-09-17T21:46:12.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316524+0000", 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043283599999999997 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316561+0000", 2024-09-17T21:46:12.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7676000000000003e-05 2024-09-17T21:46:12.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316614+0000", 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2484000000000001e-05 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1214 3.f e30/21 3:f161b17c:::rbd_data.11125c054388.0000000000000001:head v 30'23, mlcod=29'20)", 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.764610+0000", 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.77017177400000003, 2024-09-17T21:46:12.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078966290000000005, 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764610+0000", 2024-09-17T21:46:12.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764610+0000", 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764611+0000", 2024-09-17T21:46:12.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8299999999999995e-07 2024-09-17T21:46:12.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764618+0000", 2024-09-17T21:46:12.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:46:12.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764634+0000", 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6132000000000001e-05 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764643+0000", 2024-09-17T21:46:12.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2309999999999996e-06 2024-09-17T21:46:12.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764699+0000", 2024-09-17T21:46:12.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6920999999999999e-05 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764725+0000", 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5191e-05 2024-09-17T21:46:12.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772460+0000", 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0077348510000000001 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772507+0000", 2024-09-17T21:46:12.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6980000000000001e-05 2024-09-17T21:46:12.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1228 3.11 3:883b5b9d:::rbd_data.11125c054388.0000000000000008:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.810049+0000", 2024-09-17T21:46:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.724732818, 2024-09-17T21:46:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0080282370000000006, 2024-09-17T21:46:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1228 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810049+0000", 2024-09-17T21:46:12.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810049+0000", 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810050+0000", 2024-09-17T21:46:12.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8299999999999996e-07 2024-09-17T21:46:12.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810054+0000", 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9360000000000001e-06 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810065+0000", 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1534e-05 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810071+0000", 2024-09-17T21:46:12.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.648e-06 2024-09-17T21:46:12.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810116+0000", 2024-09-17T21:46:12.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5089000000000001e-05 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810170+0000", 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3732e-05 2024-09-17T21:46:12.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810285+0000", 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115246 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:12.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.817547+0000", 2024-09-17T21:46:12.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007262307 2024-09-17T21:46:12.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.818013+0000", 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046594699999999998 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.818046+0000", 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3358999999999997e-05 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.818077+0000", 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0655999999999999e-05 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1227 3.1e e30/21 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head v 30'44, mlcod=29'40)", 2024-09-17T21:46:12.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.810238+0000", 2024-09-17T21:46:12.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.72454396799999998, 2024-09-17T21:46:12.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098472379999999995, 2024-09-17T21:46:12.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810238+0000", 2024-09-17T21:46:12.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810238+0000", 2024-09-17T21:46:12.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810238+0000", 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000004e-07 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810245+0000", 2024-09-17T21:46:12.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7139999999999996e-06 2024-09-17T21:46:12.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810251+0000", 2024-09-17T21:46:12.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3149999999999997e-06 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810256+0000", 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2309999999999998e-06 2024-09-17T21:46:12.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810306+0000", 2024-09-17T21:46:12.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0417999999999997e-05 2024-09-17T21:46:12.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810327+0000", 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0917999999999999e-05 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:12.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.820054+0000", 2024-09-17T21:46:12.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097273619999999998 2024-09-17T21:46:12.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.820085+0000", 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0672e-05 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.694 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.0. 2024-09-17T21:46:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:46:12.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:12.699 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:12.693+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:12.701 INFO:teuthology.orchestra.run.smithi165.stdout:482 trunc from 0xe9efc00 to 0x1bd0000 2024-09-17T21:46:12.797 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:12.794+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:518 3.1 3.93c14a21 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:12.698752+0000", 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.14609146100000001, 2024-09-17T21:46:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.14611368499999999, 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 518 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698752+0000", 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698752+0000", 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698753+0000", 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.268e-06 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698758+0000", 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5530000000000003e-06 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698769+0000", 2024-09-17T21:46:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0971999999999999e-05 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698775+0000", 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8289999999999996e-06 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698820+0000", 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5748000000000003e-05 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:12.698828+0000", 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.515e-06 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:46:12.852 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:12.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:12.898 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:12.896+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:12.963 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:12.963 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:12.964 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:12.964 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:12.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:12.999 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:12.997+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:13.081 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:13.081 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:13.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:13.081 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8033640500000008, 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0036959290000000001, 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274236+0000", 2024-09-17T21:46:13.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274238+0000", 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.829e-06 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274245+0000", 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1810000000000004e-06 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274262+0000", 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6239000000000002e-05 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274270+0000", 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5399999999999996e-06 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274341+0000", 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1357000000000001e-05 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.274349+0000", 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7100000000000007e-06 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276472+0000", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021225319999999999 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276559+0000", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7515999999999994e-05 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.276665+0000", 2024-09-17T21:46:13.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105621 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277452+0000", 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078702500000000001 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277862+0000", 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041006899999999999 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277899+0000", 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7373000000000003e-05 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.277932+0000", 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2937e-05 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:575 3.3 3:c7f75e63:::rbd_data.1112ff4590f3.0000000000000006:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365601+0000", 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7119989369999997, 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019763160000000001, 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 575 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365601+0000", 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365601+0000", 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365602+0000", 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.37e-07 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365608+0000", 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9490000000000001e-06 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365620+0000", 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2019e-05 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365625+0000", 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0470000000000001e-06 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365676+0000", 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0812000000000001e-05 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365754+0000", 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7875999999999998e-05 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365951+0000", 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019730600000000001 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367248+0000", 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012963930000000001 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367494+0000", 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024639799999999999 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367538+0000", 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4199999999999997e-05 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367578+0000", 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9579000000000002e-05 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.096 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:576 3.9 3:94e50922:::rbd_data.1112ff4590f3.0000000000000007:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365638+0000", 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7119624580000004, 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027645220000000002, 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 576 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365638+0000", 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365638+0000", 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365638+0000", 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.46e-07 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365641+0000", 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4600000000000002e-06 2024-09-17T21:46:13.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365646+0000", 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1189999999999996e-06 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365649+0000", 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4070000000000002e-06 2024-09-17T21:46:13.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365691+0000", 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2082999999999997e-05 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365765+0000", 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4023000000000006e-05 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365946+0000", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000181077 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367758+0000", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018117789999999999 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368299+0000", 2024-09-17T21:46:13.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054044800000000001 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368360+0000", 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1554000000000001e-05 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368402+0000", 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2225999999999999e-05 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:581 3.9 3:93e4f865:::rbd_data.1112ff4590f3.000000000000000b:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365659+0000", 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7119414630000005, 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030967569999999999, 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 581 2024-09-17T21:46:13.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365659+0000", 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365659+0000", 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365659+0000", 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.72e-07 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365661+0000", 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7940000000000001e-06 2024-09-17T21:46:13.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365666+0000", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4389999999999999e-06 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365669+0000", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0419999999999998e-06 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366569+0000", 2024-09-17T21:46:13.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000899375 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366665+0000", 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5780999999999997e-05 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366859+0000", 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019435599999999999 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368074+0000", 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012150819999999999 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368704+0000", 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000630129 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368733+0000", 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8413000000000001e-05 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368756+0000", 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3074e-05 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:590 3.1a 3:59ac3078:::rbd_data.1112ff4590f3.000000000000001d:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365779+0000", 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.711821767, 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023363569999999998, 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 590 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365779+0000", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365779+0000", 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365780+0000", 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1000000000000001e-06 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365785+0000", 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1000000000000003e-06 2024-09-17T21:46:13.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365795+0000", 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0346000000000001e-05 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365804+0000", 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5820000000000002e-06 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365859+0000", 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5059999999999998e-05 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365937+0000", 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8461000000000007e-05 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366106+0000", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016840599999999999 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367521+0000", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014154930000000001 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368027+0000", 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050603900000000001 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368072+0000", 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5132000000000001e-05 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368115+0000", 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2638000000000001e-05 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:602 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.371399+0000", 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7062015370000001, 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019942829999999999, 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 602 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371399+0000", 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371399+0000", 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371400+0000", 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5999999999999991e-07 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371405+0000", 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9840000000000001e-06 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371418+0000", 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2965999999999999e-05 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371425+0000", 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371472+0000", 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7138999999999997e-05 2024-09-17T21:46:13.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371558+0000", 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5513000000000006e-05 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371916+0000", 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000358828 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373337+0000", 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001420305 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373344+0000", 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373370+0000", 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6169e-05 2024-09-17T21:46:13.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373393+0000", 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2904e-05 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:606 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.371472+0000", 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7061281959999999, 2024-09-17T21:46:13.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003035463, 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 606 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371472+0000", 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371472+0000", 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371473+0000", 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3799999999999996e-07 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371477+0000", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1960000000000001e-06 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371487+0000", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4879999999999996e-06 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371494+0000", 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7160000000000003e-06 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372324+0000", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082937099999999997 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372372+0000", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8344999999999999e-05 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372575+0000", 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020310700000000001 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373400+0000", 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082480499999999996 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374456+0000", 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010558760000000001 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374485+0000", 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9003e-05 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374508+0000", 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2818000000000001e-05 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.129 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:607 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.371511+0000", 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7060889340000003, 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031034370000000001, 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 607 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371511+0000", 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371511+0000", 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371512+0000", 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.82e-07 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371515+0000", 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9160000000000001e-06 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371522+0000", 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.041e-06 2024-09-17T21:46:13.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.371528+0000", 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3860000000000004e-06 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372837+0000", 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001308327 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372881+0000", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4821000000000003e-05 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373079+0000", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019712500000000001 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374555+0000", 2024-09-17T21:46:13.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001476046 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374564+0000", 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2369999999999998e-06 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374591+0000", 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7379e-05 2024-09-17T21:46:13.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374615+0000", 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3577000000000001e-05 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:612 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.372017+0000", 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7055836749999997, 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034591729999999998, 2024-09-17T21:46:13.136 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 612 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372017+0000", 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372017+0000", 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372018+0000", 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8800000000000001e-07 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372023+0000", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4049999999999999e-06 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372034+0000", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1430000000000001e-05 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372042+0000", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2579999999999998e-06 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373434+0000", 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013924009999999999 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373482+0000", 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8152000000000002e-05 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373672+0000", 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000190006 2024-09-17T21:46:13.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374623+0000", 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095076299999999998 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375364+0000", 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074118700000000003 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375437+0000", 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2966000000000006e-05 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375476+0000", 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8717000000000002e-05 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.142 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:613 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.372202+0000", 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7053984490000005, 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034217090000000002, 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 613 2024-09-17T21:46:13.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372202+0000", 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372202+0000", 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372203+0000", 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3699999999999999e-07 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372208+0000", 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2229999999999998e-06 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372219+0000", 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1e-05 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372237+0000", 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7867e-05 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373922+0000", 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001685071 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.373975+0000", 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2707000000000002e-05 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374183+0000", 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020825800000000001 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375518+0000", 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013354980000000001 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375535+0000", 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6543000000000001e-05 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375582+0000", 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.761e-05 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375624+0000", 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1094999999999999e-05 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:615 3.9 3:9328eab6:::rbd_object_map.1112ff4590f3:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.372574+0000", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.7050265719999995, 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034298190000000002, 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 615 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372574+0000", 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372574+0000", 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372574+0000", 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8299999999999996e-07 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372579+0000", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.34e-06 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372586+0000", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0910000000000004e-06 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.372590+0000", 2024-09-17T21:46:13.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3290000000000004e-06 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374643+0000", 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020532979999999998 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374696+0000", 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2969000000000001e-05 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.374912+0000", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021510100000000001 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375781+0000", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000869745 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375907+0000", 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126119 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.375967+0000", 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9490000000000001e-05 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.376004+0000", 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6653999999999998e-05 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:313 3.9 3:94e50922:::rbd_data.1112ff4590f3.0000000000000007:4 [sparse-read 6291456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.300274+0000", 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7773264710000003, 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001927682, 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 313 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.156 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300274+0000", 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300274+0000", 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300274+0000", 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5300000000000004e-07 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300277+0000", 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9289999999999998e-06 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300286+0000", 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3869999999999995e-06 2024-09-17T21:46:13.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300291+0000", 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7540000000000003e-06 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302093+0000", 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001801104 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302144+0000", 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1498000000000001e-05 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302202+0000", 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7457000000000002e-05 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:324 3.9 3:94e50922:::rbd_data.1112ff4590f3.0000000000000007:4 [sparse-read 7340032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.300314+0000", 2024-09-17T21:46:13.160 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7772862700000003, 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020198669999999998, 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 324 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300314+0000", 2024-09-17T21:46:13.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300314+0000", 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300315+0000", 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8100000000000003e-07 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300318+0000", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9160000000000001e-06 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300326+0000", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4379999999999995e-06 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.300332+0000", 2024-09-17T21:46:13.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6060000000000002e-06 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302222+0000", 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001890652 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302280+0000", 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302334+0000", 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4228000000000003e-05 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4480.0:340 3.1b e27/21 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head v 27'112, mlcod=27'110)", 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.306755+0000", 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7708457519999996, 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040124740000000002, 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306755+0000", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306755+0000", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306755+0000", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1200000000000002e-07 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.307962+0000", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012064459999999999 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.307972+0000", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9280000000000005e-06 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.307976+0000", 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6299999999999997e-06 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308050+0000", 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4080999999999998e-05 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308074+0000", 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3455000000000002e-05 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.310732+0000", 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026579350000000002 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.310767+0000", 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5187e-05 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:353 3.1a 3:5bb9badb:::rbd_data.1112d1429a58.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.308406+0000", 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7691944499999996, 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0090265250000000005, 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 353 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308406+0000", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308406+0000", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308407+0000", 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0999999999999997e-07 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311261+0000", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028539910000000002 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311275+0000", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4258e-05 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311283+0000", 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9840000000000001e-06 2024-09-17T21:46:13.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311341+0000", 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8186999999999997e-05 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311487+0000", 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000145895 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.311684+0000", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019694600000000001 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316129+0000", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004445137 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.317366+0000", 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012371859999999999 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.317404+0000", 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8075999999999999e-05 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.317432+0000", 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7954999999999999e-05 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4480.0:356 3.14 e27/21 3:2a14d8ba:::rbd_data.1112d1429a58.0000000000000007:head v 27'2, mlcod=0'0)", 2024-09-17T21:46:13.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.313182+0000", 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7644178669999997, 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032319089999999998, 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313182+0000", 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313182+0000", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313184+0000", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.376e-06 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313652+0000", 2024-09-17T21:46:13.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046781600000000002 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313664+0000", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2483999999999999e-05 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313674+0000", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4860000000000003e-06 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313737+0000", 2024-09-17T21:46:13.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3349000000000006e-05 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.313764+0000", 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7523999999999999e-05 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316367+0000", 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002602605 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.316414+0000", 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7268999999999998e-05 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:992 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-09-17T21:46:13.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.660627+0000", 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4169731810000004, 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019806590000000001, 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 992 2024-09-17T21:46:13.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660627+0000", 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660627+0000", 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660628+0000", 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.314e-06 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660633+0000", 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4950000000000002e-06 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660644+0000", 2024-09-17T21:46:13.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.136e-05 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660651+0000", 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660709+0000", 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8220000000000002e-05 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.660716+0000", 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7610000000000002e-06 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.662507+0000", 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017912640000000001 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.662550+0000", 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3384999999999999e-05 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.662608+0000", 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7574999999999997e-05 2024-09-17T21:46:13.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1218 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head [stat,truncate 10908160] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.774648+0000", 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.3029524379999999, 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074518040000000002, 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1218 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774648+0000", 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774648+0000", 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774650+0000", 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774657+0000", 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0169999999999999e-06 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774671+0000", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4269e-05 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774678+0000", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7229999999999998e-06 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774742+0000", 2024-09-17T21:46:13.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4240999999999997e-05 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774800+0000", 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8307000000000003e-05 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.774919+0000", 2024-09-17T21:46:13.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000118814 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.780440+0000", 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055214289999999996 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.782038+0000", 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015973269999999999 2024-09-17T21:46:13.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.782072+0000", 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4356000000000001e-05 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.782100+0000", 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7637999999999999e-05 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1220 3.1c e30/21 3:3d054346:::rbd_data.11125c054388.0000000000000004:head v 30'14, mlcod=29'11)", 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.784719+0000", 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.2928812760000001, 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051398340000000002, 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784719+0000", 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784719+0000", 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784720+0000", 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784727+0000", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1119999999999998e-06 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784739+0000", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2091e-05 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784747+0000", 2024-09-17T21:46:13.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.435e-06 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784800+0000", 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3202000000000003e-05 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784822+0000", 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2566999999999999e-05 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.789815+0000", 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049927180000000002 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.789859+0000", 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3797000000000001e-05 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.198 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1228 3.11 e30/21 3:883b5b9d:::rbd_data.11125c054388.0000000000000008:head v 30'19, mlcod=29'16)", 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.810355+0000", 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.2672457100000001, 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045531850000000004, 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810355+0000", 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810355+0000", 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810355+0000", 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3600000000000003e-07 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810362+0000", 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810373+0000", 2024-09-17T21:46:13.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0429e-05 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810380+0000", 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4919999999999996e-06 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810435+0000", 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4548e-05 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.810449+0000", 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4025999999999999e-05 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.814873+0000", 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004423728 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.814908+0000", 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5265000000000002e-05 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.205 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.205 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:13.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:13.210 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:13.207+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:13.311 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:13.308+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:13.317 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:13.317 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:13.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:13.413 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:13.411+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:13.433 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:13.433 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:13.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:13.514 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:13.512+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:13.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:13.590 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:13.590 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.351925219999998, 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037970529999999999, 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:13.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234864+0000", 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2342e-05 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234874+0000", 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0078000000000001e-05 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234894+0000", 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9208e-05 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234908+0000", 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4255e-05 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235001+0000", 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3245000000000006e-05 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235014+0000", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2692e-05 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236563+0000", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001549258 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236725+0000", 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016205000000000001 2024-09-17T21:46:13.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236884+0000", 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015916699999999999 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.237894+0000", 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010101579999999999 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238559+0000", 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000664483 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238595+0000", 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5867000000000001e-05 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238649+0000", 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4249999999999997e-05 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 20.350137891999999, 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034029759999999998, 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236642+0000", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7020000000000001e-06 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236654+0000", 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2207e-05 2024-09-17T21:46:13.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236674+0000", 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9497e-05 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236688+0000", 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.397e-05 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236772+0000", 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4654000000000006e-05 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236780+0000", 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9629999999999998e-06 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238360+0000", 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001579843 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238517+0000", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015724700000000001 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238676+0000", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015822000000000001 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239625+0000", 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094962200000000001 2024-09-17T21:46:13.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239928+0000", 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000302684 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239992+0000", 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4579000000000006e-05 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.240042+0000", 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9787999999999998e-05 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.307473586, 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042466919999999998, 2024-09-17T21:46:13.605 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279305+0000", 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.339e-06 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279310+0000", 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279317+0000", 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5059999999999998e-06 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279325+0000", 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999996e-06 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279393+0000", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8181000000000003e-05 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279400+0000", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281620+0000", 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022195130000000002 2024-09-17T21:46:13.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281751+0000", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013134099999999999 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281966+0000", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021533199999999999 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283188+0000", 2024-09-17T21:46:13.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001221895 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283464+0000", 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027606000000000001 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283508+0000", 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4026000000000002e-05 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283550+0000", 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1959000000000003e-05 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:587 3.c 3:32a591c3:::rbd_data.1112ff4590f3.0000000000000010:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.2210187269999997, 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025837009999999999, 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 587 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365760+0000", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9599999999999999e-06 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365768+0000", 2024-09-17T21:46:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8080000000000005e-06 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365795+0000", 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6339e-05 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365805+0000", 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9739999999999997e-06 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365877+0000", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2228e-05 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365979+0000", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010268999999999999 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366179+0000", 2024-09-17T21:46:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199314 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367176+0000", 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0009969200000000001 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368214+0000", 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010379370000000001 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368279+0000", 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5061999999999995e-05 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368342+0000", 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3468999999999998e-05 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:198 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3084534989999996, 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024454149999999998, 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 198 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278326+0000", 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9429999999999999e-06 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278338+0000", 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1939999999999999e-05 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278367+0000", 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9312e-05 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278374+0000", 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8940000000000004e-06 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278552+0000", 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017869199999999999 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278633+0000", 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0297999999999996e-05 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279031+0000", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039809200000000001 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280711+0000", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016796789999999999 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280718+0000", 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4209999999999998e-06 2024-09-17T21:46:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280744+0000", 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6463000000000001e-05 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280769+0000", 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4681e-05 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:208 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.307594645, 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028259169999999998, 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 208 2024-09-17T21:46:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9400000000000004e-07 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279188+0000", 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989999999999997e-06 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279197+0000", 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279206+0000", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279524+0000", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031825199999999999 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279606+0000", 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1509999999999997e-05 2024-09-17T21:46:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279842+0000", 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023666400000000001 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280777+0000", 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093496200000000001 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281955+0000", 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00117735 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281984+0000", 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9578999999999999e-05 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282008+0000", 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4164000000000001e-05 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:209 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.307558512, 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028952019999999999, 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2800000000000003e-07 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279221+0000", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.08e-06 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279226+0000", 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279229+0000", 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1489999999999998e-06 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280118+0000", 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088895399999999996 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280170+0000", 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1789000000000002e-05 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280411+0000", 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024147 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282045+0000", 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016336549999999999 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282056+0000", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1291000000000001e-05 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282084+0000", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8197000000000001e-05 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282114+0000", 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9709e-05 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:212 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3072292560000003, 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027961359999999998, 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279549+0000", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.034e-06 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279554+0000", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9620000000000001e-06 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279568+0000", 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.375e-05 2024-09-17T21:46:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279574+0000", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7209999999999997e-06 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280805+0000", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001230989 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280872+0000", 2024-09-17T21:46:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6474000000000001e-05 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281126+0000", 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025424399999999999 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282120+0000", 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099385800000000002 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282297+0000", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017720499999999999 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282324+0000", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282344+0000", 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0183999999999999e-05 2024-09-17T21:46:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:213 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3070839999999997, 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003589797, 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 213 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279694+0000", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0699999999999997e-07 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279697+0000", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5889999999999999e-06 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279705+0000", 2024-09-17T21:46:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3089999999999998e-06 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279709+0000", 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2830000000000003e-06 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281385+0000", 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001675562 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281431+0000", 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6028e-05 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281631+0000", 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020078199999999999 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282360+0000", 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072896600000000005 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283234+0000", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087365200000000004 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283262+0000", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7883999999999999e-05 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283283+0000", 2024-09-17T21:46:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1135e-05 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:218 3.c 3:32f0f6a7:::rbd_data.1112d1429a58.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1145344~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3058620540000003, 2024-09-17T21:46:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030192439999999999, 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 218 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280916+0000", 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2900000000000003e-07 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280922+0000", 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280929+0000", 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8390000000000001e-06 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280933+0000", 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0119999999999999e-06 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282393+0000", 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001459587 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282525+0000", 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000131641 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282740+0000", 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021541900000000001 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283719+0000", 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097897099999999992 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283878+0000", 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015887799999999999 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283910+0000", 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1832999999999998e-05 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283934+0000", 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4737999999999999e-05 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:333 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2848249650000003, 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002510675, 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 333 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301953+0000", 2024-09-17T21:46:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9599999999999998e-07 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301960+0000", 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301972+0000", 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2031000000000001e-05 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301980+0000", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5180000000000004e-06 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302065+0000", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4870000000000006e-05 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302154+0000", 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8937999999999994e-05 2024-09-17T21:46:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302541+0000", 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038730399999999999 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304395+0000", 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018536189999999999 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304403+0000", 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6320000000000008e-06 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304430+0000", 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7583999999999999e-05 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304463+0000", 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2675000000000001e-05 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:334 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2845473050000003, 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029186809999999998, 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 334 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302231+0000", 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9800000000000002e-07 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302235+0000", 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989999999999997e-06 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302245+0000", 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7710000000000001e-06 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302267+0000", 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1863000000000001e-05 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302922+0000", 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065509100000000001 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302980+0000", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8473000000000001e-05 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303230+0000", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024982100000000001 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304521+0000", 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00129071 2024-09-17T21:46:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305096+0000", 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057521899999999997 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305125+0000", 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8470999999999999e-05 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305149+0000", 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3864999999999999e-05 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:335 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.28436898, 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037038980000000002, 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 335 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302409+0000", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9699999999999996e-07 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302412+0000", 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:46:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302418+0000", 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.728e-06 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302422+0000", 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000001e-06 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303528+0000", 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011060219999999999 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303614+0000", 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6767000000000005e-05 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304014+0000", 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039929499999999998 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306055+0000", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002041078 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306062+0000", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306088+0000", 2024-09-17T21:46:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6333999999999999e-05 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306112+0000", 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3575999999999999e-05 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:336 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2841209359999999, 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004117674, 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 336 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302657+0000", 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6699999999999999e-07 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302661+0000", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7360000000000001e-06 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302665+0000", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0169999999999998e-06 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302669+0000", 2024-09-17T21:46:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304558+0000", 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018896 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304625+0000", 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6976000000000004e-05 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304836+0000", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021111900000000001 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306153+0000", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00131657 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306694+0000", 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054138899999999998 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306723+0000", 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8958e-05 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306774+0000", 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0711000000000003e-05 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:337 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2838860329999999, 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052613310000000002, 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 337 2024-09-17T21:46:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302892+0000", 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302894+0000", 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4770000000000002e-06 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302898+0000", 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.228e-06 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302902+0000", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4649999999999999e-06 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305188+0000", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022859769999999998 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305279+0000", 2024-09-17T21:46:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1002999999999996e-05 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305666+0000", 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038733799999999998 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308028+0000", 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023623429999999998 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308044+0000", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5967000000000001e-05 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308096+0000", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1473e-05 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308152+0000", 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6644999999999997e-05 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:338 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2832577299999999, 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049962579999999999, 2024-09-17T21:46:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 338 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303520+0000", 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1699999999999997e-07 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303524+0000", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.07e-06 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303533+0000", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9099999999999994e-06 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303538+0000", 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8180000000000003e-06 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306176+0000", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026377599999999998 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306228+0000", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2312000000000003e-05 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306449+0000", 2024-09-17T21:46:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022083399999999999 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308203+0000", 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001753976 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308467+0000", 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026413599999999999 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308492+0000", 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5324e-05 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308516+0000", 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3300999999999999e-05 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:339 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2825337450000003, 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049491889999999997, 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 339 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304244+0000", 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-17T21:46:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304248+0000", 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.053e-06 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304260+0000", 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2057e-05 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304265+0000", 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0769999999999998e-06 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306814+0000", 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025484209999999999 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306896+0000", 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1982000000000002e-05 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.307420+0000", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052378600000000004 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308525+0000", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001105672 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309107+0000", 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058199 2024-09-17T21:46:13.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309162+0000", 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4435999999999999e-05 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309193+0000", 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0892000000000002e-05 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1214 3.f 3:f161b17c:::rbd_data.11125c054388.0000000000000001:head [stat,truncate 12582912] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.8224936860000001, 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.008423692, 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1214 2024-09-17T21:46:13.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764285+0000", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764291+0000", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8109999999999999e-06 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764303+0000", 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2517000000000001e-05 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764309+0000", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1430000000000003e-06 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764361+0000", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1604999999999998e-05 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764417+0000", 2024-09-17T21:46:13.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6227999999999998e-05 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764532+0000", 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115293 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.770673+0000", 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061402590000000003 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772640+0000", 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019671490000000001 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772674+0000", 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4579000000000002e-05 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772707+0000", 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2824000000000003e-05 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1220 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.802499316, 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083268920000000007, 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1220 2024-09-17T21:46:13.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784280+0000", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.809e-06 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784284+0000", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784297+0000", 2024-09-17T21:46:13.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2133000000000001e-05 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784306+0000", 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3260000000000002e-06 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784360+0000", 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4660000000000002e-05 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784452+0000", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1821000000000001e-05 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784634+0000", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018206099999999999 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792045+0000", 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074103349999999997 2024-09-17T21:46:13.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792504+0000", 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045927000000000001 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792557+0000", 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3399999999999997e-05 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792605+0000", 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7355999999999999e-05 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1222 3.2 e30/21 3:4025fe03:::rbd_data.11125c054388.0000000000000005:head v 30'21, mlcod=29'18)", 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.7914927710000002, 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052176820000000004, 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:13.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:13.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795285+0000", 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.73e-07 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795288+0000", 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8539999999999999e-06 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795298+0000", 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0012e-05 2024-09-17T21:46:13.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795304+0000", 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795355+0000", 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0727999999999999e-05 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:13.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795369+0000", 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3582e-05 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.800460+0000", 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050912880000000002 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.800502+0000", 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2116999999999997e-05 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:13.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:13.717 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:13.718 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:13.719 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:13.720 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:13.716+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:13.720 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:13.720 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:13.736 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:13.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:13.753 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:13.753 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:13.770 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:13.771 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:13.821 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:13.819+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:13.828 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:13.828 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:13.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:13.923 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:13.920+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:13.944 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:13.944 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:13.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:14.025 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:14.023+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:14.060 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.060 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:14.153 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:14.151+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:14.217 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.217 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:14.266 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:14.264+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:14.328 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:14.326+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:14.372 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.373 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:14.429 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:14.426+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.488 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:14.530 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:14.528+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:14.603 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.604 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:14.656 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:14.654+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:14.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:46:14.726 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.726 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.732 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:14.729+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:14.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:14.833 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:14.831+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:14.855 INFO:teuthology.orchestra.run.smithi165.stdout:485 read 0x9b8000 thru 0x9c6f8a (0xef8b bytes) 2024-09-17T21:46:14.885 INFO:teuthology.orchestra.run.smithi165.stdout:488 write 0x96b8600 thru 0x96bb825 (0x3226 bytes) 2024-09-17T21:46:14.885 INFO:teuthology.orchestra.run.smithi165.stdout:492 read 0x1ad5000 thru 0x1adba5e (0x6a5f bytes) 2024-09-17T21:46:14.886 INFO:teuthology.orchestra.run.smithi165.stdout:493 write 0x2493200 thru 0x249dc73 (0xaa74 bytes) 2024-09-17T21:46:14.900 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:14.900 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:14.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:14.918 INFO:teuthology.orchestra.run.smithi165.stdout:494 write 0xaae0400 thru 0xaae0e6d (0xa6e bytes) 2024-09-17T21:46:14.937 INFO:teuthology.orchestra.run.smithi165.stdout:497 write 0xe0b3e00 thru 0xe0c08f2 (0xcaf3 bytes) 2024-09-17T21:46:14.938 INFO:teuthology.orchestra.run.smithi165.stdout:498 read 0x451d400 thru 0x45240d4 (0x6cd5 bytes) 2024-09-17T21:46:14.938 INFO:teuthology.orchestra.run.smithi165.stdout:500 read 0x8668a00 thru 0x8674632 (0xbc33 bytes) 2024-09-17T21:46:14.938 INFO:teuthology.orchestra.run.smithi165.stdout:504 read 0x3600000 thru 0x3600e58 (0xe59 bytes) 2024-09-17T21:46:14.950 INFO:teuthology.orchestra.run.smithi165.stdout:507 trunc from 0xe0c0a00 to 0x452e200 2024-09-17T21:46:14.952 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:14.950+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:15.010 INFO:teuthology.orchestra.run.smithi165.stdout:510 write 0x51bc800 thru 0x51bf828 (0x3029 bytes) 2024-09-17T21:46:15.015 INFO:teuthology.orchestra.run.smithi165.stdout:516 trunc from 0x51bfa00 to 0x851d400 2024-09-17T21:46:15.028 INFO:teuthology.orchestra.run.smithi165.stdout:517 punch from 0x1759800 to 0x1760800, (0x7000 bytes) 2024-09-17T21:46:15.035 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:15.033+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:15.044 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.044 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:15.134 INFO:teuthology.orchestra.run.smithi165.stdout:518 write 0xdd70a00 thru 0xdd7aa49 (0xa04a bytes) 2024-09-17T21:46:15.134 INFO:teuthology.orchestra.run.smithi165.stdout:520 write 0x7372e00 thru 0x73799f2 (0x6bf3 bytes) 2024-09-17T21:46:15.135 INFO:teuthology.orchestra.run.smithi165.stdout:521 punch from 0x86ffe00 to 0x8706600, (0x6800 bytes) 2024-09-17T21:46:15.137 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:15.134+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:15.141 INFO:teuthology.orchestra.run.smithi165.stdout:522 write 0xca26000 thru 0xca3209d (0xc09e bytes) 2024-09-17T21:46:15.141 INFO:teuthology.orchestra.run.smithi165.stdout:524 read 0x7ee9c00 thru 0x7ef475b (0xab5c bytes) 2024-09-17T21:46:15.141 INFO:teuthology.orchestra.run.smithi165.stdout:525 write 0x1709400 thru 0x1711780 (0x8381 bytes) 2024-09-17T21:46:15.142 INFO:teuthology.orchestra.run.smithi165.stdout:528 read 0x43f6c00 thru 0x44060c7 (0xf4c8 bytes) 2024-09-17T21:46:15.142 INFO:teuthology.orchestra.run.smithi165.stdout:530 write 0xacfe600 thru 0xad0a593 (0xbf94 bytes) 2024-09-17T21:46:15.154 INFO:teuthology.orchestra.run.smithi165.stdout:531 trunc from 0xdd7ac00 to 0x3a00000 2024-09-17T21:46:15.209 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.209 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:15.237 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:15.235+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:15.278 INFO:teuthology.orchestra.run.smithi165.stdout:534 read 0x1118e00 thru 0x111aba3 (0x1da4 bytes) 2024-09-17T21:46:15.280 INFO:teuthology.orchestra.run.smithi165.stdout:535 read 0x2731e00 thru 0x27332b4 (0x14b5 bytes) 2024-09-17T21:46:15.281 INFO:teuthology.orchestra.run.smithi165.stdout:536 read 0x8d1400 thru 0x8db472 (0xa073 bytes) 2024-09-17T21:46:15.303 INFO:teuthology.orchestra.run.smithi165.stdout:537 write 0xc972600 thru 0xc976eb7 (0x48b8 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:538 write 0xb985000 thru 0xb9902c1 (0xb2c2 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:540 read 0x2379000 thru 0x23887a3 (0xf7a4 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:541 read 0x6998c00 thru 0x699be94 (0x3295 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:544 read 0x6a3e000 thru 0x6a40168 (0x2169 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:547 write 0x3d30e00 thru 0x3d3aae6 (0x9ce7 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:554 read 0xb870000 thru 0xb876a8a (0x6a8b bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:556 write 0x6caac00 thru 0x6cb22f7 (0x76f8 bytes) 2024-09-17T21:46:15.305 INFO:teuthology.orchestra.run.smithi165.stdout:557 read 0xdd0600 thru 0xdd5986 (0x5387 bytes) 2024-09-17T21:46:15.306 INFO:teuthology.orchestra.run.smithi165.stdout:558 read 0x2975000 thru 0x297c469 (0x746a bytes) 2024-09-17T21:46:15.307 INFO:teuthology.orchestra.run.smithi165.stdout:563 read 0x3e63e00 thru 0x3e6dfb5 (0xa1b6 bytes) 2024-09-17T21:46:15.307 INFO:teuthology.orchestra.run.smithi165.stdout:567 read 0x2ba7a00 thru 0x2bb498c (0xcf8d bytes) 2024-09-17T21:46:15.307 INFO:teuthology.orchestra.run.smithi165.stdout:571 write 0x20d2200 thru 0x20d2dcf (0xbd0 bytes) 2024-09-17T21:46:15.307 INFO:teuthology.orchestra.run.smithi165.stdout:572 punch from 0xad07000 to 0xad0b800, (0x4800 bytes) 2024-09-17T21:46:15.308 INFO:teuthology.orchestra.run.smithi165.stdout:573 write 0x8991000 thru 0x899eafa (0xdafb bytes) 2024-09-17T21:46:15.308 INFO:teuthology.orchestra.run.smithi165.stdout:574 punch from 0x37d400 to 0x38a400, (0xd000 bytes) 2024-09-17T21:46:15.338 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:15.336+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:15.368 INFO:teuthology.orchestra.run.smithi165.stdout:575 read 0x667b800 thru 0x667b99f (0x1a0 bytes) 2024-09-17T21:46:15.368 INFO:teuthology.orchestra.run.smithi165.stdout:576 write 0x78bec00 thru 0x78ce3a8 (0xf7a9 bytes) 2024-09-17T21:46:15.368 INFO:teuthology.orchestra.run.smithi165.stdout:579 read 0x2853600 thru 0x285b389 (0x7d8a bytes) 2024-09-17T21:46:15.419 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.419 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:15.439 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:15.436+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:15.477 INFO:teuthology.orchestra.run.smithi165.stdout:580 write 0x1057600 thru 0x1061347 (0x9d48 bytes) 2024-09-17T21:46:15.479 INFO:teuthology.orchestra.run.smithi165.stdout:581 write 0x8cae200 thru 0x8cb3dba (0x5bbb bytes) 2024-09-17T21:46:15.479 INFO:teuthology.orchestra.run.smithi165.stdout:582 write 0x64ee200 thru 0x64fa21d (0xc01e bytes) 2024-09-17T21:46:15.485 INFO:teuthology.orchestra.run.smithi165.stdout:586 trunc from 0xc977000 to 0x7913a00 2024-09-17T21:46:15.539 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:15.537+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:15.631 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.631 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.639 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:15.637+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:15.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:15.665 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:46:15.684 INFO:teuthology.orchestra.run.smithi165.stdout:588 trunc from 0x7913a00 to 0xd89cc00 2024-09-17T21:46:15.741 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:15.738+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:15.757 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.757 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:15.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:15.851 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:15.848+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:15.905 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:15.905 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:15.912 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:15.913 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:15.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:15.922 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:15.923 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:15.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:15.938 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:15.955 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:15.956 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:15.962 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:15.960+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:15.972 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:15.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:15.989 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:16.043 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:16.041+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:16.069 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:16.069 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:16.069 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:16.069 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:16.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:16.144 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:16.142+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:16.184 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:16.184 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:16.184 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:16.184 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:16.184 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:16.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:16.276 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:16.274+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.103495432999999, 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037970529999999999, 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234852+0000", 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234864+0000", 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2342e-05 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234874+0000", 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0078000000000001e-05 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234894+0000", 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9208e-05 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.234908+0000", 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4255e-05 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235001+0000", 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3245000000000006e-05 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.235014+0000", 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2692e-05 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236563+0000", 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001549258 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236725+0000", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016205000000000001 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.236884+0000", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015916699999999999 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.237894+0000", 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010101579999999999 2024-09-17T21:46:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238559+0000", 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000664483 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238595+0000", 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5867000000000001e-05 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:52.238649+0000", 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4249999999999997e-05 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4211.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.101708105, 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034029759999999998, 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4211", 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.7:0/3952673826", 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 5 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236639+0000", 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236642+0000", 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7020000000000001e-06 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236654+0000", 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2207e-05 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236674+0000", 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9497e-05 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236688+0000", 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.397e-05 2024-09-17T21:46:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236772+0000", 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4654000000000006e-05 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.236780+0000", 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9629999999999998e-06 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238360+0000", 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001579843 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238517+0000", 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015724700000000001 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.238676+0000", 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015822000000000001 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239625+0000", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094962200000000001 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239928+0000", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000302684 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.239992+0000", 2024-09-17T21:46:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4579000000000006e-05 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:53.240042+0000", 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9787999999999998e-05 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.059043798999999, 2024-09-17T21:46:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042466919999999998, 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279304+0000", 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279305+0000", 2024-09-17T21:46:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.339e-06 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279310+0000", 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279317+0000", 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5059999999999998e-06 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279325+0000", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4019999999999996e-06 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279393+0000", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8181000000000003e-05 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.279400+0000", 2024-09-17T21:46:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281620+0000", 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022195130000000002 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281751+0000", 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013134099999999999 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.281966+0000", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021533199999999999 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283188+0000", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001221895 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283464+0000", 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027606000000000001 2024-09-17T21:46:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283508+0000", 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4026000000000002e-05 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:03.283550+0000", 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1959000000000003e-05 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4465.0:587 3.c 3:32a591c3:::rbd_data.1112ff4590f3.0000000000000010:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.9725889399999996, 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025837009999999999, 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4465", 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1435487200", 2024-09-17T21:46:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 587 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365758+0000", 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365760+0000", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9599999999999999e-06 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365768+0000", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8080000000000005e-06 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365795+0000", 2024-09-17T21:46:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6339e-05 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365805+0000", 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9739999999999997e-06 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365877+0000", 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2228e-05 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.365979+0000", 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010268999999999999 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.366179+0000", 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000199314 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.367176+0000", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0009969200000000001 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368214+0000", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010379370000000001 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368279+0000", 2024-09-17T21:46:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5061999999999995e-05 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:07.368342+0000", 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3468999999999998e-05 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:198 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0600237119999996, 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024454149999999998, 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 198 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278324+0000", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278326+0000", 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9429999999999999e-06 2024-09-17T21:46:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278338+0000", 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1939999999999999e-05 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278367+0000", 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9312e-05 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278374+0000", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8940000000000004e-06 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278552+0000", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017869199999999999 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.278633+0000", 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0297999999999996e-05 2024-09-17T21:46:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279031+0000", 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039809200000000001 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280711+0000", 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016796789999999999 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280718+0000", 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4209999999999998e-06 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280744+0000", 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6463000000000001e-05 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280769+0000", 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4681e-05 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:208 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0591648580000008, 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028259169999999998, 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 208 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279183+0000", 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9400000000000004e-07 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279188+0000", 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989999999999997e-06 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279197+0000", 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8710000000000003e-06 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279206+0000", 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3719999999999994e-06 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279524+0000", 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031825199999999999 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279606+0000", 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1509999999999997e-05 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279842+0000", 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023666400000000001 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280777+0000", 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093496200000000001 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281955+0000", 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00117735 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281984+0000", 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9578999999999999e-05 2024-09-17T21:46:16.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282008+0000", 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4164000000000001e-05 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:209 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:16.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0591287250000008, 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028952019999999999, 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 209 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279219+0000", 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2800000000000003e-07 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279221+0000", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.08e-06 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279226+0000", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279229+0000", 2024-09-17T21:46:16.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1489999999999998e-06 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280118+0000", 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088895399999999996 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280170+0000", 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1789000000000002e-05 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280411+0000", 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024147 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282045+0000", 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016336549999999999 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282056+0000", 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1291000000000001e-05 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282084+0000", 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8197000000000001e-05 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282114+0000", 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9709e-05 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:212 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0587994690000002, 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027961359999999998, 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 212 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279548+0000", 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279549+0000", 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.034e-06 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279554+0000", 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9620000000000001e-06 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279568+0000", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.375e-05 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279574+0000", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7209999999999997e-06 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280805+0000", 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001230989 2024-09-17T21:46:16.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280872+0000", 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6474000000000001e-05 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281126+0000", 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025424399999999999 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282120+0000", 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099385800000000002 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282297+0000", 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017720499999999999 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282324+0000", 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6715000000000001e-05 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282344+0000", 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0183999999999999e-05 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:213 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0586542130000005, 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003589797, 2024-09-17T21:46:16.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 213 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:16.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279693+0000", 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279694+0000", 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0699999999999997e-07 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279697+0000", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5889999999999999e-06 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279705+0000", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3089999999999998e-06 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.279709+0000", 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2830000000000003e-06 2024-09-17T21:46:16.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281385+0000", 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001675562 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281431+0000", 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6028e-05 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.281631+0000", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020078199999999999 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282360+0000", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072896600000000005 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283234+0000", 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087365200000000004 2024-09-17T21:46:16.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283262+0000", 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7883999999999999e-05 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283283+0000", 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1135e-05 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:218 3.c 3:32f0f6a7:::rbd_data.1112d1429a58.0000000000000012:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 1145344~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0574322669999994, 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030192439999999999, 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 218 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280915+0000", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280916+0000", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2900000000000003e-07 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280922+0000", 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280929+0000", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8390000000000001e-06 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.280933+0000", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0119999999999999e-06 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282393+0000", 2024-09-17T21:46:16.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001459587 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282525+0000", 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000131641 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.282740+0000", 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021541900000000001 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283719+0000", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097897099999999992 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283878+0000", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015887799999999999 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283910+0000", 2024-09-17T21:46:16.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1832999999999998e-05 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.283934+0000", 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4737999999999999e-05 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:333 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0363951779999994, 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002510675, 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 333 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301952+0000", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301953+0000", 2024-09-17T21:46:16.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9599999999999998e-07 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301960+0000", 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6080000000000001e-06 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301972+0000", 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2031000000000001e-05 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.301980+0000", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5180000000000004e-06 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302065+0000", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4870000000000006e-05 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302154+0000", 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8937999999999994e-05 2024-09-17T21:46:16.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302541+0000", 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038730399999999999 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304395+0000", 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018536189999999999 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304403+0000", 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6320000000000008e-06 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304430+0000", 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7583999999999999e-05 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304463+0000", 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2675000000000001e-05 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:334 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0361175179999993, 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029186809999999998, 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 334 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302230+0000", 2024-09-17T21:46:16.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302231+0000", 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9800000000000002e-07 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302235+0000", 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3989999999999997e-06 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302245+0000", 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7710000000000001e-06 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302267+0000", 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1863000000000001e-05 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302922+0000", 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00065509100000000001 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302980+0000", 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8473000000000001e-05 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303230+0000", 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024982100000000001 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304521+0000", 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00129071 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305096+0000", 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057521899999999997 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305125+0000", 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8470999999999999e-05 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305149+0000", 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3864999999999999e-05 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:335 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0359391930000008, 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037038980000000002, 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 335 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302408+0000", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302409+0000", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9699999999999996e-07 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302412+0000", 2024-09-17T21:46:16.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302418+0000", 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.728e-06 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302422+0000", 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6600000000000001e-06 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303528+0000", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011060219999999999 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303614+0000", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6767000000000005e-05 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304014+0000", 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039929499999999998 2024-09-17T21:46:16.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306055+0000", 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002041078 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306062+0000", 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4970000000000004e-06 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306088+0000", 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6333999999999999e-05 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306112+0000", 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3575999999999999e-05 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:336 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0356911489999998, 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004117674, 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 336 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302656+0000", 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302657+0000", 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6699999999999999e-07 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302661+0000", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7360000000000001e-06 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302665+0000", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0169999999999998e-06 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302669+0000", 2024-09-17T21:46:16.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304558+0000", 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018896 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304625+0000", 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6976000000000004e-05 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304836+0000", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021111900000000001 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306153+0000", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00131657 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306694+0000", 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054138899999999998 2024-09-17T21:46:16.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306723+0000", 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8958e-05 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306774+0000", 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0711000000000003e-05 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:337 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0354562460000007, 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052613310000000002, 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 337 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302891+0000", 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302892+0000", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302894+0000", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4770000000000002e-06 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302898+0000", 2024-09-17T21:46:16.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.228e-06 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.302902+0000", 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4649999999999999e-06 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305188+0000", 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022859769999999998 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305279+0000", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1002999999999996e-05 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.305666+0000", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038733799999999998 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308028+0000", 2024-09-17T21:46:16.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023623429999999998 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308044+0000", 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5967000000000001e-05 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308096+0000", 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1473e-05 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308152+0000", 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6644999999999997e-05 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:338 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0348279429999998, 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049962579999999999, 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 338 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303519+0000", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303520+0000", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1699999999999997e-07 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303524+0000", 2024-09-17T21:46:16.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.07e-06 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303533+0000", 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9099999999999994e-06 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.303538+0000", 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8180000000000003e-06 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306176+0000", 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026377599999999998 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306228+0000", 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2312000000000003e-05 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306449+0000", 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022083399999999999 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308203+0000", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001753976 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308467+0000", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026413599999999999 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308492+0000", 2024-09-17T21:46:16.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5324e-05 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308516+0000", 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3300999999999999e-05 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4480.0:339 3.c 3:305b8c40:::rbd_object_map.1112d1429a58:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:16.448 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.0341039579999993, 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0049491889999999997, 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4480", 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 339 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304243+0000", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304244+0000", 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.23e-07 2024-09-17T21:46:16.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304248+0000", 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.053e-06 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304260+0000", 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2057e-05 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.304265+0000", 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0769999999999998e-06 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306814+0000", 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025484209999999999 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.306896+0000", 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1982000000000002e-05 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.307420+0000", 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052378600000000004 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.308525+0000", 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001105672 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309107+0000", 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058199 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309162+0000", 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4435999999999999e-05 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:08.309193+0000", 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0892000000000002e-05 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1214 3.f 3:f161b17c:::rbd_data.11125c054388.0000000000000001:head [stat,truncate 12582912] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5740638990000004, 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.008423692, 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:16.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1214 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764283+0000", 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764285+0000", 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764291+0000", 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8109999999999999e-06 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764303+0000", 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2517000000000001e-05 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764309+0000", 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1430000000000003e-06 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764361+0000", 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1604999999999998e-05 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764417+0000", 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6227999999999998e-05 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.764532+0000", 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115293 2024-09-17T21:46:16.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.770673+0000", 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0061402590000000003 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772640+0000", 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019671490000000001 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772674+0000", 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4579000000000002e-05 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.772707+0000", 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2824000000000003e-05 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4542.0:1220 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:16.461 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5540695289999995, 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0083268920000000007, 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4542", 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1220 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784278+0000", 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784280+0000", 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.809e-06 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784284+0000", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7210000000000002e-06 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784297+0000", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2133000000000001e-05 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784306+0000", 2024-09-17T21:46:16.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3260000000000002e-06 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784360+0000", 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4660000000000002e-05 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784452+0000", 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1821000000000001e-05 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.784634+0000", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018206099999999999 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792045+0000", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0074103349999999997 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792504+0000", 2024-09-17T21:46:16.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045927000000000001 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792557+0000", 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3399999999999997e-05 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.792605+0000", 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7355999999999999e-05 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4542.0:1222 3.2 e30/21 3:4025fe03:::rbd_data.11125c054388.0000000000000005:head v 30'21, mlcod=29'18)", 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5430629839999996, 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052176820000000004, 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:16.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795284+0000", 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795285+0000", 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.73e-07 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795288+0000", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8539999999999999e-06 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795298+0000", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0012e-05 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795304+0000", 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:46:16.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795355+0000", 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0727999999999999e-05 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.795369+0000", 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3582e-05 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:16.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.800460+0000", 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050912880000000002 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:11.800502+0000", 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2116999999999997e-05 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:16.472 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:16.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:16.474 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:16.471+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:16.575 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:16.572+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:16.590 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:16.590 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:16.590 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:16.590 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:16.599 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:16.676 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:16.673+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:16.686 INFO:teuthology.orchestra.run.smithi165.stdout:589 read 0xb358000 thru 0xb35eee8 (0x6ee9 bytes) 2024-09-17T21:46:16.687 INFO:teuthology.orchestra.run.smithi165.stdout:590 read 0x6578600 thru 0x657a480 (0x1e81 bytes) 2024-09-17T21:46:16.688 INFO:teuthology.orchestra.run.smithi165.stdout:591 write 0x153ea00 thru 0x154e80e (0xfe0f bytes) 2024-09-17T21:46:16.689 INFO:teuthology.orchestra.run.smithi165.stdout:592 read 0xacdee00 thru 0xace497d (0x5b7e bytes) 2024-09-17T21:46:16.689 INFO:teuthology.orchestra.run.smithi165.stdout:593 punch from 0x3068a00 to 0x3073c00, (0xb200 bytes) 2024-09-17T21:46:16.691 INFO:teuthology.orchestra.run.smithi165.stdout:594 write 0xd7f6800 thru 0xd800452 (0x9c53 bytes) 2024-09-17T21:46:16.691 INFO:teuthology.orchestra.run.smithi165.stdout:596 read 0xce5b600 thru 0xce658bc (0xa2bd bytes) 2024-09-17T21:46:16.691 INFO:teuthology.orchestra.run.smithi165.stdout:599 write 0x5f7e000 thru 0x5f7fc56 (0x1c57 bytes) 2024-09-17T21:46:16.691 INFO:teuthology.orchestra.run.smithi165.stdout:600 read 0x6a11000 thru 0x6a1438e (0x338f bytes) 2024-09-17T21:46:16.705 INFO:teuthology.orchestra.run.smithi165.stdout:601 trunc from 0xd89cc00 to 0x4d0ae00 2024-09-17T21:46:16.719 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:16.719 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:16.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:16.766 INFO:teuthology.orchestra.run.smithi165.stdout:602 read 0x1045c00 thru 0x10498e3 (0x3ce4 bytes) 2024-09-17T21:46:16.776 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:16.774+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:603 write 0x9a39400 thru 0x9a3def2 (0x4af3 bytes) 2024-09-17T21:46:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:604 read 0x6131600 thru 0x613ea28 (0xd429 bytes) 2024-09-17T21:46:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:606 read 0x44af400 thru 0x44b50fb (0x5cfc bytes) 2024-09-17T21:46:16.788 INFO:teuthology.orchestra.run.smithi165.stdout:607 write 0x40e4c00 thru 0x40e968b (0x4a8c bytes) 2024-09-17T21:46:16.797 INFO:teuthology.orchestra.run.smithi165.stdout:609 trunc from 0x9a3e000 to 0x2c1fe00 2024-09-17T21:46:16.845 INFO:teuthology.orchestra.run.smithi165.stdout:610 trunc from 0x2c1fe00 to 0x7acd600 2024-09-17T21:46:16.850 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:16.850 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:16.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:16.850 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5634671190000002, 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:16.851 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:16.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:16.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:16.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:16.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:16.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:16.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:16.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:16.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:16.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:16.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:16.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:16.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:16.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5415403150000007, 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:16.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:16.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:16.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:16.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:16.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:16.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:16.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:16.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:16.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:16.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:16.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:16.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.1901522790000003, 2024-09-17T21:46:16.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:16.881 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:16.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:16.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6852231609999997, 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:16.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:16.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6852134249999997, 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:16.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6852038140000003, 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6851929830000003, 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:16.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:16.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6851793920000002, 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:16.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6851690420000001, 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:412 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24522752~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166181+0000", 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6800801219999997, 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030942750000000001, 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 412 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166181+0000", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166181+0000", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166182+0000", 2024-09-17T21:46:16.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7700000000000004e-07 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166184+0000", 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1169999999999998e-06 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166191+0000", 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166197+0000", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6930000000000004e-06 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169185+0000", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002988479 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169224+0000", 2024-09-17T21:46:16.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8606999999999998e-05 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169275+0000", 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1471000000000002e-05 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:16.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6800566650000004, 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:16.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:462 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24764416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170520+0000", 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6757414449999999, 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031665479999999999, 2024-09-17T21:46:16.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 462 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170520+0000", 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170520+0000", 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170520+0000", 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170521+0000", 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.347e-06 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170525+0000", 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6109999999999998e-06 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170528+0000", 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.774e-06 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173581+0000", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030530319999999998 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173630+0000", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9020000000000002e-05 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173686+0000", 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6415000000000001e-05 2024-09-17T21:46:16.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6757299699999999, 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:16.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6755907610000005, 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:16.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6755737709999998, 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:16.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6755527729999997, 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:512 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24801280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175544+0000", 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6707168750000001, 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031154469999999999, 2024-09-17T21:46:16.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 512 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175544+0000", 2024-09-17T21:46:16.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175544+0000", 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175545+0000", 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175546+0000", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.088e-06 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175549+0000", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175551+0000", 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6199999999999999e-06 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178558+0000", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030068130000000001 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178605+0000", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6477999999999998e-05 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178660+0000", 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5220000000000003e-05 2024-09-17T21:46:16.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6707066189999997, 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:16.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6706942099999997, 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:16.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.6706838780000002, 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:16.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:16.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:16.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:16.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:16.970 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:16.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:16.973 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:16.969+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:16.973 INFO:teuthology.orchestra.run.smithi165.stdout:612 read 0x79b3400 thru 0x79c04ab (0xd0ac bytes) 2024-09-17T21:46:16.973 INFO:teuthology.orchestra.run.smithi165.stdout:613 read 0x3b65a00 thru 0x3b69de4 (0x43e5 bytes) 2024-09-17T21:46:16.973 INFO:teuthology.orchestra.run.smithi165.stdout:614 write 0x786a000 thru 0x78713a7 (0x73a8 bytes) 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:615 trunc from 0x7acd600 to 0x6f38c00 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:617 write 0x1329400 thru 0x13384fa (0xf0fb bytes) 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:619 read 0x69a0e00 thru 0x69a3ec2 (0x30c3 bytes) 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:621 read 0x4f57600 thru 0x4f58b49 (0x154a bytes) 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:623 write 0x26fee00 thru 0x2704160 (0x5361 bytes) 2024-09-17T21:46:16.974 INFO:teuthology.orchestra.run.smithi165.stdout:624 flatten 2024-09-17T21:46:17.072 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:17.070+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:17.080 INFO:teuthology.orchestra.run.smithi165.stdout:625 write 0x76b9000 thru 0x76b9b13 (0xb14 bytes) 2024-09-17T21:46:17.080 INFO:teuthology.orchestra.run.smithi165.stdout:626 write 0x28e9e00 thru 0x28f5da6 (0xbfa7 bytes) 2024-09-17T21:46:17.080 INFO:teuthology.orchestra.run.smithi165.stdout:630 read 0x331de00 thru 0x3324dfe (0x6fff bytes) 2024-09-17T21:46:17.087 INFO:teuthology.orchestra.run.smithi165.stdout:631 trunc from 0x76b9c00 to 0xd21f800 2024-09-17T21:46:17.108 INFO:teuthology.orchestra.run.smithi165.stdout:634 trunc from 0xd21f800 to 0xa26d400 2024-09-17T21:46:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:17.126 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:17.126 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:17.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:17.140 INFO:teuthology.orchestra.run.smithi165.stdout:635 write 0xbbc0a00 thru 0xbbcdd4d (0xd34e bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:636 write 0xd895a00 thru 0xd8a354c (0xdb4d bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:637 read 0x8c20800 thru 0x8c2203f (0x1840 bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:638 write 0xbaa4000 thru 0xbab1947 (0xd948 bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:639 read 0x5aff600 thru 0x5b0c319 (0xcd1a bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:640 write 0x212a200 thru 0x2131e77 (0x7c78 bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:641 write 0x93e4c00 thru 0x93e709c (0x249d bytes) 2024-09-17T21:46:17.173 INFO:teuthology.orchestra.run.smithi165.stdout:643 read 0x9a1da00 thru 0x9a24f02 (0x7503 bytes) 2024-09-17T21:46:17.174 INFO:teuthology.orchestra.run.smithi165.stdout:644 read 0xba04800 thru 0xba12071 (0xd872 bytes) 2024-09-17T21:46:17.174 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:17.170+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:17.174 INFO:teuthology.orchestra.run.smithi165.stdout:646 punch from 0x9d1f000 to 0x9d28c00, (0x9c00 bytes) 2024-09-17T21:46:17.175 INFO:teuthology.orchestra.run.smithi165.stdout:648 punch from 0x71db800 to 0x71dec00, (0x3400 bytes) 2024-09-17T21:46:17.175 INFO:teuthology.orchestra.run.smithi165.stdout:649 read 0x7d08600 thru 0x7d0d96f (0x5370 bytes) 2024-09-17T21:46:17.180 INFO:teuthology.orchestra.run.smithi165.stdout:650 trunc from 0xd8a3600 to 0x8c73200 2024-09-17T21:46:17.251 INFO:teuthology.orchestra.run.smithi165.stdout:653 read 0x1573a00 thru 0x1580311 (0xc912 bytes) 2024-09-17T21:46:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:656 read 0x8974c00 thru 0x8977050 (0x2451 bytes) 2024-09-17T21:46:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:657 punch from 0x53eb000 to 0x53f8c00, (0xdc00 bytes) 2024-09-17T21:46:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:660 punch from 0xfdee00 to 0xfece00, (0xe000 bytes) 2024-09-17T21:46:17.274 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:17.271+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:17.282 INFO:teuthology.orchestra.run.smithi165.stdout:661 write 0xde9d000 thru 0xdeaa632 (0xd633 bytes) 2024-09-17T21:46:17.302 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:17.302 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:17.302 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:17.302 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:17.302 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:17.305 INFO:teuthology.orchestra.run.smithi165.stdout:664 write 0xeaaf000 thru 0xeab8cf0 (0x9cf1 bytes) 2024-09-17T21:46:17.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:17.312 INFO:teuthology.orchestra.run.smithi165.stdout:665 write 0xc70c00 thru 0xc7b726 (0xab27 bytes) 2024-09-17T21:46:17.312 INFO:teuthology.orchestra.run.smithi165.stdout:666 read 0xc01a200 thru 0xc023c5a (0x9a5b bytes) 2024-09-17T21:46:17.312 INFO:teuthology.orchestra.run.smithi165.stdout:667 punch from 0x79cae00 to 0x79d0c00, (0x5e00 bytes) 2024-09-17T21:46:17.313 INFO:teuthology.orchestra.run.smithi165.stdout:672 write 0x5ffc200 thru 0x5ffea85 (0x2886 bytes) 2024-09-17T21:46:17.313 INFO:teuthology.orchestra.run.smithi165.stdout:673 read 0x6c6f600 thru 0x6c76b29 (0x752a bytes) 2024-09-17T21:46:17.313 INFO:teuthology.orchestra.run.smithi165.stdout:674 write 0x657d000 thru 0x658788c (0xa88d bytes) 2024-09-17T21:46:17.323 INFO:teuthology.orchestra.run.smithi165.stdout:675 trunc from 0xeab8e00 to 0x5b9be00 2024-09-17T21:46:17.374 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:17.372+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:17.391 INFO:teuthology.orchestra.run.smithi165.stdout:678 trunc from 0x5b9be00 to 0xbd49a00 2024-09-17T21:46:17.401 INFO:teuthology.orchestra.run.smithi165.stdout:679 read 0x7eade00 thru 0x7eb841c (0xa61d bytes) 2024-09-17T21:46:17.402 INFO:teuthology.orchestra.run.smithi165.stdout:680 write 0x3923000 thru 0x392d1d9 (0xa1da bytes) 2024-09-17T21:46:17.408 INFO:teuthology.orchestra.run.smithi165.stdout:681 trunc from 0xbd49a00 to 0x6e52800 2024-09-17T21:46:17.429 INFO:teuthology.orchestra.run.smithi165.stdout:683 read 0x4bf6c00 thru 0x4c015e1 (0xa9e2 bytes) 2024-09-17T21:46:17.430 INFO:teuthology.orchestra.run.smithi165.stdout:684 write 0x6186c00 thru 0x618afe4 (0x43e5 bytes) 2024-09-17T21:46:17.431 INFO:teuthology.orchestra.run.smithi165.stdout:685 read 0x65dc200 thru 0x65e4c93 (0x8a94 bytes) 2024-09-17T21:46:17.431 INFO:teuthology.orchestra.run.smithi165.stdout:686 write 0x37600 thru 0x439da (0xc3db bytes) 2024-09-17T21:46:17.431 INFO:teuthology.orchestra.run.smithi165.stdout:690 write 0x175f400 thru 0x1761ce3 (0x28e4 bytes) 2024-09-17T21:46:17.432 INFO:teuthology.orchestra.run.smithi165.stdout:691 read 0x3fc0a00 thru 0x3fce57b (0xdb7c bytes) 2024-09-17T21:46:17.436 INFO:teuthology.orchestra.run.smithi165.stdout:693 trunc from 0x6e52800 to 0x3e41600 2024-09-17T21:46:17.439 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53170570900000003, 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:17.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53169540800000004, 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:17.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.449 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.531679334, 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:17.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:17.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53166812100000005, 2024-09-17T21:46:17.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53165068299999996, 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:17.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:17.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53163981299999996, 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:17.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.53144493599999998, 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52974026600000002, 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:17.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:17.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52972962800000001, 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:17.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52971953500000002, 2024-09-17T21:46:17.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:17.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52970941599999999, 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:17.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:17.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:17.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52757918599999998, 2024-09-17T21:46:17.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:17.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52756945499999996, 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:17.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52678609600000004, 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:17.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:17.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52677052599999996, 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:17.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:17.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52667762399999996, 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:17.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:17.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52618300500000004, 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:17.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:17.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:17.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52603896000000006, 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:17.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:17.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.52535352300000004, 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:17.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.524991133, 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:17.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.532 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:694 write 0xf7d000 thru 0xf8af0d (0xdf0e bytes) 2024-09-17T21:46:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:696 read 0x328b400 thru 0x3296848 (0xb449 bytes) 2024-09-17T21:46:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:697 read 0x306d600 thru 0x3072d33 (0x5734 bytes) 2024-09-17T21:46:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:698 write 0xa226400 thru 0xa230653 (0xa254 bytes) 2024-09-17T21:46:17.534 INFO:teuthology.orchestra.run.smithi165.stdout:700 read 0x242ee00 thru 0x242f0f1 (0x2f2 bytes) 2024-09-17T21:46:17.535 INFO:teuthology.orchestra.run.smithi165.stdout:702 trunc from 0xa230800 to 0x1bb0000 2024-09-17T21:46:17.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:17.535 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:17.531+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:17.570 INFO:teuthology.orchestra.run.smithi165.stdout:703 write 0xac41a00 thru 0xac4fd5e (0xe35f bytes) 2024-09-17T21:46:17.575 INFO:teuthology.orchestra.run.smithi165.stdout:704 write 0x4f07600 thru 0x4f0d00e (0x5a0f bytes) 2024-09-17T21:46:17.576 INFO:teuthology.orchestra.run.smithi165.stdout:707 trunc from 0xac4fe00 to 0xa79d400 2024-09-17T21:46:17.611 INFO:teuthology.orchestra.run.smithi165.stdout:709 punch from 0xafe800 to 0xb05e00, (0x7600 bytes) 2024-09-17T21:46:17.617 INFO:teuthology.orchestra.run.smithi165.stdout:712 write 0x4eba00 thru 0x4f3779 (0x7d7a bytes) 2024-09-17T21:46:17.618 INFO:teuthology.orchestra.run.smithi165.stdout:713 read 0x9550600 thru 0x9554289 (0x3c8a bytes) 2024-09-17T21:46:17.619 INFO:teuthology.orchestra.run.smithi165.stdout:714 trunc from 0xa79d400 to 0x9418000 2024-09-17T21:46:17.634 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:17.631+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:17.645 INFO:teuthology.orchestra.run.smithi165.stdout:718 trunc from 0x9418000 to 0x15dc000 2024-09-17T21:46:17.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:46:17.692 INFO:teuthology.orchestra.run.smithi165.stdout:723 write 0xcd67c00 thru 0xcd734bb (0xb8bc bytes) 2024-09-17T21:46:17.693 INFO:teuthology.orchestra.run.smithi165.stdout:724 punch from 0x76bbe00 to 0x76c4a00, (0x8c00 bytes) 2024-09-17T21:46:17.694 INFO:teuthology.orchestra.run.smithi165.stdout:725 punch from 0x2769a00 to 0x2779400, (0xfa00 bytes) 2024-09-17T21:46:17.694 INFO:teuthology.orchestra.run.smithi165.stdout:726 punch from 0x6304000 to 0x6307e00, (0x3e00 bytes) 2024-09-17T21:46:17.694 INFO:teuthology.orchestra.run.smithi165.stdout:727 write 0x1c4a600 thru 0x1c55069 (0xaa6a bytes) 2024-09-17T21:46:17.694 INFO:teuthology.orchestra.run.smithi165.stdout:730 write 0x11f8000 thru 0x1202561 (0xa562 bytes) 2024-09-17T21:46:17.695 INFO:teuthology.orchestra.run.smithi165.stdout:732 punch from 0x16f1600 to 0x16f9e00, (0x8800 bytes) 2024-09-17T21:46:17.695 INFO:teuthology.orchestra.run.smithi165.stdout:734 write 0x2123e00 thru 0x212688b (0x2a8c bytes) 2024-09-17T21:46:17.695 INFO:teuthology.orchestra.run.smithi165.stdout:735 punch from 0x677fc00 to 0x6789600, (0x9a00 bytes) 2024-09-17T21:46:17.696 INFO:teuthology.orchestra.run.smithi165.stdout:736 write 0xb594400 thru 0xb59dd8a (0x998b bytes) 2024-09-17T21:46:17.696 INFO:teuthology.orchestra.run.smithi165.stdout:truncating to largest ever: 0xebb0200 2024-09-17T21:46:17.698 INFO:teuthology.orchestra.run.smithi165.stdout:738 trunc from 0xcd73600 to 0xebb0200 2024-09-17T21:46:17.700 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:17.700 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:17.701 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-17T21:46:17.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T21:46:17.703 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:14386 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'735, mlcod=35'730)", 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.701881+0000", 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8628999999999994e-05, 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000114651, 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:17.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701881+0000", 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701881+0000", 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:17.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701882+0000", 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701885+0000", 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4999999999999999e-06 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701894+0000", 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0070000000000003e-06 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701899+0000", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2969999999999997e-06 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701942+0000", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2942999999999999e-05 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.701952+0000", 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0744000000000001e-05 2024-09-17T21:46:17.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:46:17.708 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:17.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:17.734 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:17.732+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:17.783 INFO:teuthology.orchestra.run.smithi165.stdout:739 trunc from 0xebb0200 to 0xa273a00 2024-09-17T21:46:17.812 INFO:teuthology.orchestra.run.smithi165.stdout:741 punch from 0x23d4e00 to 0x23df000, (0xa200 bytes) 2024-09-17T21:46:17.814 INFO:teuthology.orchestra.run.smithi165.stdout:742 read 0x6868800 thru 0x6868dbd (0x5be bytes) 2024-09-17T21:46:17.814 INFO:teuthology.orchestra.run.smithi165.stdout:743 punch from 0x79e9c00 to 0x79f2a00, (0x8e00 bytes) 2024-09-17T21:46:17.815 INFO:teuthology.orchestra.run.smithi165.stdout:744 punch from 0x43f2000 to 0x43f6800, (0x4800 bytes) 2024-09-17T21:46:17.815 INFO:teuthology.orchestra.run.smithi165.stdout:746 write 0x9c38000 thru 0x9c3c6d3 (0x46d4 bytes) 2024-09-17T21:46:17.820 INFO:teuthology.orchestra.run.smithi165.stdout:751 trunc from 0xa273a00 to 0xe8eb400 2024-09-17T21:46:17.835 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:17.833+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:17.852 INFO:teuthology.orchestra.run.smithi165.stdout:753 read 0x2e07800 thru 0x2e10b6a (0x936b bytes) 2024-09-17T21:46:17.852 INFO:teuthology.orchestra.run.smithi165.stdout:755 write 0x89ac000 thru 0x89b52a1 (0x92a2 bytes) 2024-09-17T21:46:17.867 INFO:teuthology.orchestra.run.smithi165.stdout:756 trunc from 0xe8eb400 to 0x1f0cc00 2024-09-17T21:46:17.893 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:17.893 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:17.893 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:17.905 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:17.931 INFO:teuthology.orchestra.run.smithi165.stdout:758 read 0x1a5c000 thru 0x1a6661f (0xa620 bytes) 2024-09-17T21:46:17.936 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:17.934+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:17.947 INFO:teuthology.orchestra.run.smithi165.stdout:759 write 0x6323c00 thru 0x6324406 (0x807 bytes) 2024-09-17T21:46:17.948 INFO:teuthology.orchestra.run.smithi165.stdout:763 read 0x53b6800 thru 0x53c23c8 (0xbbc9 bytes) 2024-09-17T21:46:17.948 INFO:teuthology.orchestra.run.smithi165.stdout:764 read 0x1445000 thru 0x1451fac (0xcfad bytes) 2024-09-17T21:46:17.949 INFO:teuthology.orchestra.run.smithi165.stdout:765 punch from 0x50b4c00 to 0x50c0800, (0xbc00 bytes) 2024-09-17T21:46:17.950 INFO:teuthology.orchestra.run.smithi165.stdout:771 trunc from 0x6324600 to 0x5521000 2024-09-17T21:46:17.982 INFO:teuthology.orchestra.run.smithi165.stdout:772 punch from 0x4454c00 to 0x4456600, (0x1a00 bytes) 2024-09-17T21:46:17.983 INFO:teuthology.orchestra.run.smithi165.stdout:774 read 0xd85200 thru 0xd8b8da (0x66db bytes) 2024-09-17T21:46:17.984 INFO:teuthology.orchestra.run.smithi165.stdout:776 punch from 0x3a38a00 to 0x3a3d000, (0x4600 bytes) 2024-09-17T21:46:17.984 INFO:teuthology.orchestra.run.smithi165.stdout:778 punch from 0x39a7c00 to 0x39ada00, (0x5e00 bytes) 2024-09-17T21:46:17.984 INFO:teuthology.orchestra.run.smithi165.stdout:779 read 0xb25000 thru 0xb3388c (0xe88d bytes) 2024-09-17T21:46:17.992 INFO:teuthology.orchestra.run.smithi165.stdout:781 punch from 0x352be00 to 0x3539800, (0xda00 bytes) 2024-09-17T21:46:18.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:18.018 INFO:teuthology.orchestra.run.smithi165.stdout:785 write 0x7104c00 thru 0x710e4a0 (0x98a1 bytes) 2024-09-17T21:46:18.019 INFO:teuthology.orchestra.run.smithi165.stdout:786 write 0x274c600 thru 0x2755d49 (0x974a bytes) 2024-09-17T21:46:18.019 INFO:teuthology.orchestra.run.smithi165.stdout:787 read 0x4d3c200 thru 0x4d4a633 (0xe434 bytes) 2024-09-17T21:46:18.024 INFO:teuthology.orchestra.run.smithi165.stdout:788 trunc from 0x710e600 to 0x2e00000 2024-09-17T21:46:18.037 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:18.034+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:18.085 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:18.085 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4542.0:1218 3.9 e30/21 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head v 30'67, mlcod=29'64)", 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.3068662890000002, 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067374460000000002, 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775156+0000", 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:46:18.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775164+0000", 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775185+0000", 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0710999999999999e-05 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775197+0000", 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1739000000000001e-05 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775269+0000", 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775290+0000", 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0866999999999999e-05 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781826+0000", 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065362390000000001 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781893+0000", 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6446999999999996e-05 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2720975259999996, 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:18.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:18.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:18.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:18.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11451 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079614002, 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053110780000000003, 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11451 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002409+0000", 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002413+0000", 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4029999999999999e-06 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002415+0000", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007559+0000", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005143173 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007608+0000", 2024-09-17T21:46:18.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9521999999999998e-05 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007719+0000", 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011066299999999999 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11452 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:18.101 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.07960251, 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054903180000000001, 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11452 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:18.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:18.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002424+0000", 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002431+0000", 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.905e-06 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002434+0000", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.083e-06 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007757+0000", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053229310000000004 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007805+0000", 2024-09-17T21:46:18.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8090000000000002e-05 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007909+0000", 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104514 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11453 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1515520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079583991, 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056581649999999997, 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11453 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4400000000000001e-07 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002439+0000", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3740000000000001e-06 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002443+0000", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9519999999999996e-06 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002448+0000", 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9239999999999998e-06 2024-09-17T21:46:18.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007946+0000", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054980029999999996 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007992+0000", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008096+0000", 2024-09-17T21:46:18.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103478 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11454 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1564672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079570092, 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058292750000000001, 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11454 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002453+0000", 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.28e-06 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002457+0000", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3529999999999997e-06 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002460+0000", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008133+0000", 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056725229999999996 2024-09-17T21:46:18.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008178+0000", 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5463999999999997e-05 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008281+0000", 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102674 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11455 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079558292, 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060027099999999996, 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11455 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002464+0000", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-07 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002465+0000", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.234e-06 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002469+0000", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0759999999999996e-06 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002472+0000", 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008318+0000", 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058459009999999997 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008364+0000", 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5948999999999997e-05 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008466+0000", 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010257200000000001 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11457 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079546865, 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061758150000000003, 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11457 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002476+0000", 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2109999999999999e-06 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002480+0000", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002484+0000", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.253e-06 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008503+0000", 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060196650000000004 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008548+0000", 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5223999999999999e-05 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008651+0000", 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102143 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11458 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0795332479999999, 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063602019999999997, 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11458 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002489+0000", 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:18.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002491+0000", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.968e-06 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002498+0000", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002503+0000", 2024-09-17T21:46:18.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.254e-06 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008688+0000", 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061850450000000001 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008745+0000", 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6647999999999997e-05 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008849+0000", 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010412900000000001 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11459 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079511906, 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065278610000000003, 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11459 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002513+0000", 2024-09-17T21:46:18.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3590000000000002e-06 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002525+0000", 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2245e-05 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002531+0000", 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008887+0000", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063556790000000004 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008934+0000", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7157000000000003e-05 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009038+0000", 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103892 2024-09-17T21:46:18.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11460 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079483065, 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066803280000000001, 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11460 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:18.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002542+0000", 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002551+0000", 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6229999999999992e-06 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002557+0000", 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009068+0000", 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065103599999999998 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009115+0000", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7818999999999997e-05 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009219+0000", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010344999999999999 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11461 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1859584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:18.138 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0794516080000001, 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068348430000000002, 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11461 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002571+0000", 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002573+0000", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002582+0000", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2809999999999997e-06 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002588+0000", 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9819999999999996e-06 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009256+0000", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066681739999999998 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009303+0000", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6663999999999999e-05 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009405+0000", 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101753 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11462 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1908736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079418808, 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006995823, 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11462 2024-09-17T21:46:18.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:18.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002606+0000", 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.807e-06 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002614+0000", 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8259999999999993e-06 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002620+0000", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8869999999999997e-06 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009442+0000", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068217820000000002 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009497+0000", 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5600000000000003e-05 2024-09-17T21:46:18.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009599+0000", 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101369 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11463 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1957888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0793940799999999, 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071654559999999997, 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11463 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002631+0000", 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-17T21:46:18.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002638+0000", 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9909999999999996e-06 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002643+0000", 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009636+0000", 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069926459999999999 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009682+0000", 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6406000000000003e-05 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009793+0000", 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011097799999999999 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11464 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.0793719230000001, 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073216799999999997, 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11464 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002652+0000", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002659+0000", 2024-09-17T21:46:18.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8569999999999997e-06 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002665+0000", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.789e-06 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009824+0000", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071584190000000001 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009871+0000", 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.782e-05 2024-09-17T21:46:18.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009971+0000", 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100076 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11465 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.079347668, 2024-09-17T21:46:18.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074631050000000003, 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11465 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002681+0000", 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002690+0000", 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5639999999999997e-06 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002711+0000", 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009992+0000", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072811359999999997 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010037+0000", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4721000000000001e-05 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010137+0000", 2024-09-17T21:46:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100447 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15737 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'905, mlcod=35'900)", 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.054444748000000001, 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052510500000000002, 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027583+0000", 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.716e-06 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027601+0000", 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7606000000000001e-05 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027610+0000", 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027668+0000", 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8301999999999998e-05 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027687+0000", 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8567e-05 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032787+0000", 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050997359999999997 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032828+0000", 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1248999999999998e-05 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15743 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'911, mlcod=35'903)", 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.049351632999999999, 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052577359999999998, 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032671+0000", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899999999999995e-07 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032676+0000", 2024-09-17T21:46:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032686+0000", 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-05 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032693+0000", 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032952+0000", 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025868400000000002 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032976+0000", 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3995e-05 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037203+0000", 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042275050000000003 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037928+0000", 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000724411 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15901 3.18 3:18be054b:::rbd_data.11121f8e87bb.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.028539087000000001, 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054083960000000002, 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15901 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9400000000000005e-07 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053488+0000", 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2910000000000001e-06 2024-09-17T21:46:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053497+0000", 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6800000000000005e-06 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053504+0000", 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053558+0000", 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3987000000000003e-05 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053679+0000", 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120191 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053977+0000", 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029810399999999999 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.056472+0000", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024953789999999998 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058804+0000", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002332373 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058855+0000", 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.024e-05 2024-09-17T21:46:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058891+0000", 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6359000000000002e-05 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15903 3.7 3:e5ee49b6:::rbd_data.11121f8e87bb.0000000000000067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:18.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.028332033, 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055705099999999999, 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15903 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:18.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6499999999999999e-07 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053699+0000", 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054218+0000", 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051917300000000005 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054227+0000", 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0270000000000001e-06 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054307+0000", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0543000000000005e-05 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054381+0000", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3633999999999993e-05 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054482+0000", 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010119300000000001 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.055778+0000", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012961120000000001 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059175+0000", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033961880000000001 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059226+0000", 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1353000000000001e-05 2024-09-17T21:46:18.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059260+0000", 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4289000000000003e-05 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:18.182 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:18.184 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:18.181+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:18.185 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:18.185 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:18.105+0000 7f0c28614640 -1 mon.a@0(leader).osd e35 definitely_dead 0 2024-09-17T21:46:18.185 INFO:teuthology.orchestra.run.smithi165.stdout:789 write 0xd7e1000 thru 0xd7e3c8d (0x2c8e bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:792 write 0xc171e00 thru 0xc17b42c (0x962d bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:796 write 0xc251000 thru 0xc25f0d1 (0xe0d2 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:798 write 0xe160a00 thru 0xe16a940 (0x9f41 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:799 write 0x265a400 thru 0x265ca86 (0x2687 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:801 punch from 0x48cae00 to 0x48cee00, (0x4000 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:802 punch from 0x2d93c00 to 0x2da1600, (0xda00 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:804 punch from 0x2209400 to 0x2213600, (0xa200 bytes) 2024-09-17T21:46:18.186 INFO:teuthology.orchestra.run.smithi165.stdout:805 trunc from 0xe16aa00 to 0x7f17600 2024-09-17T21:46:18.192 INFO:teuthology.orchestra.run.smithi165.stdout:806 punch from 0x110f000 to 0x110fa00, (0xa00 bytes) 2024-09-17T21:46:18.194 INFO:teuthology.orchestra.run.smithi165.stdout:810 write 0x34acc00 thru 0x34adcec (0x10ed bytes) 2024-09-17T21:46:18.215 INFO:teuthology.orchestra.run.smithi165.stdout:811 write 0xe0bbe00 thru 0xe0bc10e (0x30f bytes) 2024-09-17T21:46:18.215 INFO:teuthology.orchestra.run.smithi165.stdout:812 punch from 0x418c400 to 0x4198a00, (0xc600 bytes) 2024-09-17T21:46:18.217 INFO:teuthology.orchestra.run.smithi165.stdout:817 write 0x90ae000 thru 0x90bbfb1 (0xdfb2 bytes) 2024-09-17T21:46:18.217 INFO:teuthology.orchestra.run.smithi165.stdout:818 clone 6 order 22 su 32768 sc 15 2024-09-17T21:46:18.284 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:18.282+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:18.305 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:18.305 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:18.305 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:18.313 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:18.369 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:18.386 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:18.398 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:18.396+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:18.403 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:18.439 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:18.463 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:18.463 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:18.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:18.485 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:18.483+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:18.586 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:18.584+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:18.620 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:18.620 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:18.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.0. 2024-09-17T21:46:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:46:18.692 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-17T21:46:18.712 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:18.710+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:18.779 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:18.779 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:18.787 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:18.788 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:18.785+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:18.889 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:18.886+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:18.934 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:18.934 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:18.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:18.989 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:18.987+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:19.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.049 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:19.101 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:19.099+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:19.166 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.166 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:19.190 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:19.188+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:19.291 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:19.288+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:19.321 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.322 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.330 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:19.392 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:19.390+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:19.477 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.477 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:19.493 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:19.490+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:19.593 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:19.591+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:19.607 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.607 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:19.701 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:19.699+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:19.793 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.793 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.795 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:19.793+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:19.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:19.896 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:19.893+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:19.950 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:19.950 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:19.959 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:19.996 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:19.994+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:20.066 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:20.066 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:20.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:20.097 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:20.095+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:20.181 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:20.182 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:20.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:20.235 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:20.233+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:20.298 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:20.298 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:20.299 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:20.297+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:20.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:20.400 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:20.398+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:20.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:20.454 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:20.454 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:20.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:20.504 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:20.502+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:20.573 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:20.574 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:20.591 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:20.591 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:20.603 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:20.600+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:20.608 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:20.608 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:20.612 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:20.612 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:20.612 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:20.612 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:20.612 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:20.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:20.626 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:20.626 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:20.643 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:20.643 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:20.661 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:20.661 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:20.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:46:20.713 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:20.711+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:20.735 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:20.735 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:20.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:20.735 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:20.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4542.0:1218 3.9 e30/21 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head v 30'67, mlcod=29'64)", 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9567733650000001, 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067374460000000002, 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:20.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775156+0000", 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:46:20.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775164+0000", 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775185+0000", 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0710999999999999e-05 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775197+0000", 2024-09-17T21:46:20.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1739000000000001e-05 2024-09-17T21:46:20.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775269+0000", 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775290+0000", 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0866999999999999e-05 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781826+0000", 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065362390000000001 2024-09-17T21:46:20.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781893+0000", 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6446999999999996e-05 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9220046019999995, 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:20.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:20.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11451 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7295210779999999, 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053110780000000003, 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11451 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T21:46:20.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002409+0000", 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002413+0000", 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4029999999999999e-06 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002415+0000", 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007559+0000", 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005143173 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007608+0000", 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9521999999999998e-05 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007719+0000", 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011066299999999999 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11452 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7295095859999998, 2024-09-17T21:46:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054903180000000001, 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11452 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002424+0000", 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002431+0000", 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.905e-06 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002434+0000", 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.083e-06 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007757+0000", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053229310000000004 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007805+0000", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8090000000000002e-05 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007909+0000", 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104514 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11453 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1515520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7294910670000001, 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056581649999999997, 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11453 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4400000000000001e-07 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002439+0000", 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3740000000000001e-06 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002443+0000", 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9519999999999996e-06 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002448+0000", 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9239999999999998e-06 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007946+0000", 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054980029999999996 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007992+0000", 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008096+0000", 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103478 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11454 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1564672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7294771679999998, 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058292750000000001, 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11454 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002453+0000", 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.28e-06 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002457+0000", 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3529999999999997e-06 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002460+0000", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008133+0000", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056725229999999996 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008178+0000", 2024-09-17T21:46:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5463999999999997e-05 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008281+0000", 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102674 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11455 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7294653680000001, 2024-09-17T21:46:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060027099999999996, 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11455 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002464+0000", 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-07 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002465+0000", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.234e-06 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002469+0000", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0759999999999996e-06 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002472+0000", 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-17T21:46:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008318+0000", 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058459009999999997 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008364+0000", 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5948999999999997e-05 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008466+0000", 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010257200000000001 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11457 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7294539410000001, 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061758150000000003, 2024-09-17T21:46:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11457 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002476+0000", 2024-09-17T21:46:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2109999999999999e-06 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002480+0000", 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002484+0000", 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.253e-06 2024-09-17T21:46:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008503+0000", 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060196650000000004 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008548+0000", 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5223999999999999e-05 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008651+0000", 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102143 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11458 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.729440324, 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063602019999999997, 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11458 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002489+0000", 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002491+0000", 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.968e-06 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002498+0000", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002503+0000", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.254e-06 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008688+0000", 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061850450000000001 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008745+0000", 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6647999999999997e-05 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008849+0000", 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010412900000000001 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11459 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7294189819999999, 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065278610000000003, 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11459 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002513+0000", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3590000000000002e-06 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002525+0000", 2024-09-17T21:46:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2245e-05 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002531+0000", 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008887+0000", 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063556790000000004 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008934+0000", 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7157000000000003e-05 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009038+0000", 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103892 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11460 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7293901410000001, 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066803280000000001, 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11460 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002542+0000", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002551+0000", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6229999999999992e-06 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002557+0000", 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009068+0000", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065103599999999998 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009115+0000", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7818999999999997e-05 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009219+0000", 2024-09-17T21:46:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010344999999999999 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11461 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1859584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7293586840000001, 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068348430000000002, 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11461 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002571+0000", 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002573+0000", 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002582+0000", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2809999999999997e-06 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002588+0000", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9819999999999996e-06 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009256+0000", 2024-09-17T21:46:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066681739999999998 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009303+0000", 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6663999999999999e-05 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009405+0000", 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101753 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11462 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1908736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7293258840000001, 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006995823, 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11462 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002606+0000", 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.807e-06 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002614+0000", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8259999999999993e-06 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002620+0000", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8869999999999997e-06 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009442+0000", 2024-09-17T21:46:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068217820000000002 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009497+0000", 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5600000000000003e-05 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009599+0000", 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101369 2024-09-17T21:46:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11463 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1957888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.729301156, 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071654559999999997, 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11463 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002631+0000", 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002638+0000", 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9909999999999996e-06 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002643+0000", 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009636+0000", 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069926459999999999 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009682+0000", 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6406000000000003e-05 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009793+0000", 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011097799999999999 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11464 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7292789989999999, 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073216799999999997, 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11464 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002652+0000", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002659+0000", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8569999999999997e-06 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002665+0000", 2024-09-17T21:46:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.789e-06 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009824+0000", 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071584190000000001 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009871+0000", 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.782e-05 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009971+0000", 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100076 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11465 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7292547439999999, 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074631050000000003, 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11465 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002681+0000", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002690+0000", 2024-09-17T21:46:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5639999999999997e-06 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002711+0000", 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009992+0000", 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072811359999999997 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010037+0000", 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4721000000000001e-05 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010137+0000", 2024-09-17T21:46:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100447 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15737 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'905, mlcod=35'900)", 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.7043518239999997, 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052510500000000002, 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027583+0000", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.716e-06 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027601+0000", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7606000000000001e-05 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027610+0000", 2024-09-17T21:46:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027668+0000", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8301999999999998e-05 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027687+0000", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8567e-05 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032787+0000", 2024-09-17T21:46:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050997359999999997 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032828+0000", 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1248999999999998e-05 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15743 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'911, mlcod=35'903)", 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.699258709, 2024-09-17T21:46:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052577359999999998, 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032671+0000", 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899999999999995e-07 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032676+0000", 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032686+0000", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-05 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032693+0000", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032952+0000", 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025868400000000002 2024-09-17T21:46:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032976+0000", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3995e-05 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037203+0000", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042275050000000003 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037928+0000", 2024-09-17T21:46:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000724411 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15901 3.18 3:18be054b:::rbd_data.11121f8e87bb.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.6784461630000003, 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054083960000000002, 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15901 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9400000000000005e-07 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053488+0000", 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2910000000000001e-06 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053497+0000", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6800000000000005e-06 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053504+0000", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053558+0000", 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3987000000000003e-05 2024-09-17T21:46:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053679+0000", 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120191 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053977+0000", 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029810399999999999 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.056472+0000", 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024953789999999998 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058804+0000", 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002332373 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058855+0000", 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.024e-05 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058891+0000", 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6359000000000002e-05 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15903 3.7 3:e5ee49b6:::rbd_data.11121f8e87bb.0000000000000067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.6782391089999997, 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055705099999999999, 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15903 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6499999999999999e-07 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053699+0000", 2024-09-17T21:46:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054218+0000", 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051917300000000005 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054227+0000", 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0270000000000001e-06 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054307+0000", 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0543000000000005e-05 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054381+0000", 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3633999999999993e-05 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054482+0000", 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010119300000000001 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.055778+0000", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012961120000000001 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059175+0000", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033961880000000001 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059226+0000", 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1353000000000001e-05 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059260+0000", 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4289000000000003e-05 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:20.836 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:20.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:20.838 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:20.835+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:20.938 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:20.936+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:20.967 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:20.967 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:20.967 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:20.968 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:20.977 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:21.039 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:21.036+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:21.085 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:21.085 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:21.085 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:21.085 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:21.085 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:21.094 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:21.177 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:21.175+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:21.241 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:21.238+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:21.244 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.082167114000001, 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11508 3.11 3:883b5b9d:::rbd_data.11125c054388.0000000000000008:7 [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2375887409999997, 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011205596, 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11508 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2600000000000001e-07 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003385+0000", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003388+0000", 2024-09-17T21:46:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5590000000000001e-06 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003391+0000", 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014342+0000", 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010951106 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014423+0000", 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1283000000000001e-05 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014585+0000", 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016200700000000001 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:2636 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [watch reconnect cookie 140714606129008 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8840117169999999, 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011916717, 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2636 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.63e-07 2024-09-17T21:46:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356964+0000", 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4880000000000004e-06 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356969+0000", 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3800000000000002e-06 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356972+0000", 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367098+0000", 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010125861 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367173+0000", 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4714000000000003e-05 2024-09-17T21:46:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.368874+0000", 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017007179999999999 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:12952 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8839547649999999, 2024-09-17T21:46:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011082310999999999, 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 12952 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357015+0000", 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357019+0000", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357026+0000", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4280000000000001e-06 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357030+0000", 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3150000000000002e-06 2024-09-17T21:46:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367922+0000", 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010892208 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367999+0000", 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7173000000000001e-05 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.368096+0000", 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6699000000000004e-05 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13672 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671385088] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.721725433, 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011128726, 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13672 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519244+0000", 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519245+0000", 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519249+0000", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4460000000000001e-06 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519252+0000", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.061e-06 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530172+0000", 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010919836 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530238+0000", 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6672000000000001e-05 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530372+0000", 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133511 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13673 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633670968352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7217116319999999, 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011284531, 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13673 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.39e-07 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519259+0000", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.508e-06 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519262+0000", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1540000000000002e-06 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519265+0000", 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5950000000000001e-06 2024-09-17T21:46:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530412+0000", 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011146937000000001 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530456+0000", 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4070000000000003e-05 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530542+0000", 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5927999999999995e-05 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13674 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671395568] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7216820369999999, 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011500078, 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13674 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519289+0000", 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7320000000000001e-06 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519292+0000", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3139999999999999e-06 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519295+0000", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6759999999999999e-06 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530580+0000", 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011285117000000001 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530654+0000", 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4134999999999994e-05 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530787+0000", 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013265400000000001 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13675 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671407856] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7216705239999999, 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011670329, 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13675 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519300+0000", 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.404e-06 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519303+0000", 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519306+0000", 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6579999999999998e-06 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530831+0000", 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011525844 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530876+0000", 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4885999999999997e-05 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530968+0000", 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2119000000000004e-05 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13676 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671128672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.721650645, 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011888612, 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13676 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519320+0000", 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.857e-06 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519325+0000", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5519999999999998e-06 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519328+0000", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.88e-06 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531010+0000", 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011682203 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531076+0000", 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5748999999999997e-05 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531207+0000", 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000130958 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13677 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671428000] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.7216260930000002, 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012027163, 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13677 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519347+0000", 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9879999999999998e-06 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519351+0000", 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519353+0000", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7750000000000001e-06 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531247+0000", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011893124999999999 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531288+0000", 2024-09-17T21:46:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1931000000000002e-05 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531370+0000", 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1216000000000003e-05 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13678 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671149968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.721596216, 2024-09-17T21:46:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012240881, 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13678 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519373+0000", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519375+0000", 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-17T21:46:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519378+0000", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3610000000000001e-06 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519381+0000", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7010000000000001e-06 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531414+0000", 2024-09-17T21:46:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012033333 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531482+0000", 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7958000000000002e-05 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531613+0000", 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013111000000000001 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15175 3.15 3:ae5b5468:::rbd_data.11121f8e87bb.00000000000000f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3484912950000001, 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011304611000000001, 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15175 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892478+0000", 2024-09-17T21:46:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7599999999999997e-07 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892481+0000", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4949999999999998e-06 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892486+0000", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6849999999999998e-06 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892489+0000", 2024-09-17T21:46:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902702+0000", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01021262 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902774+0000", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1501999999999994e-05 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902870+0000", 2024-09-17T21:46:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6459000000000006e-05 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903650+0000", 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077978100000000001 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903743+0000", 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3430000000000005e-05 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903764+0000", 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0769999999999999e-05 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903782+0000", 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7915e-05 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15228 3.15 3:ab518994:::rbd_data.11121f8e87bb.0000000000000141:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3419973380000001, 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011106143000000001, 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15228 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898972+0000", 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898974+0000", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3999999999999999e-06 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898981+0000", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.387e-06 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898985+0000", 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908707+0000", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097212989999999992 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908765+0000", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8559e-05 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908842+0000", 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6676999999999998e-05 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.909594+0000", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075257399999999999 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910038+0000", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000443831 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910059+0000", 2024-09-17T21:46:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1189999999999999e-05 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910077+0000", 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8011000000000001e-05 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3002 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3400135880000001, 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011091225999999999, 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3002 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900956+0000", 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.269e-06 2024-09-17T21:46:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900961+0000", 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900972+0000", 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0575e-05 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900979+0000", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.911925+0000", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010946035 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.911977+0000", 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1873000000000003e-05 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912046+0000", 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9536000000000006e-05 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15245 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.339896655, 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011088077999999999, 2024-09-17T21:46:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15245 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901073+0000", 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901076+0000", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3220000000000001e-06 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901083+0000", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.148e-06 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901087+0000", 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0509999999999998e-06 2024-09-17T21:46:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912075+0000", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010987393999999999 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912113+0000", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8392000000000001e-05 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912160+0000", 2024-09-17T21:46:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7104999999999997e-05 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3003 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.339882834, 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011227009, 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3003 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901087+0000", 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4900000000000005e-07 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901090+0000", 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7060000000000002e-06 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901098+0000", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4610000000000002e-06 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901102+0000", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0679999999999998e-06 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912183+0000", 2024-09-17T21:46:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011081336000000001 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912219+0000", 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5679000000000002e-05 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912313+0000", 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.391e-05 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15246 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3397436530000002, 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011271723000000001, 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15246 2024-09-17T21:46:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901226+0000", 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901228+0000", 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901234+0000", 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.755e-06 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901238+0000", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912346+0000", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011108487 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912415+0000", 2024-09-17T21:46:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8547999999999995e-05 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912497+0000", 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1805999999999996e-05 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3004 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.339712902, 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011465227999999999, 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3004 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901257+0000", 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-07 2024-09-17T21:46:21.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901260+0000", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1099999999999999e-06 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901266+0000", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9139999999999996e-06 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901269+0000", 2024-09-17T21:46:21.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7900000000000001e-06 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912527+0000", 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011258068 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912586+0000", 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9086000000000003e-05 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912721+0000", 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013446500000000001 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3017 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3353312829999999, 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011080370000000001, 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3017 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905638+0000", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6199999999999997e-07 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905641+0000", 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3390000000000001e-06 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905648+0000", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8399999999999997e-06 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905652+0000", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1219999999999997e-06 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916608+0000", 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010955998999999999 2024-09-17T21:46:21.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916652+0000", 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-05 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916718+0000", 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5518000000000005e-05 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3018 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.3351788390000001, 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011167348000000001, 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3018 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.338 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905791+0000", 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:46:21.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905795+0000", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4410000000000003e-06 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905805+0000", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905812+0000", 2024-09-17T21:46:21.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9310000000000002e-06 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916840+0000", 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011028532000000001 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916882+0000", 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1523999999999997e-05 2024-09-17T21:46:21.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916957+0000", 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5378999999999998e-05 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.342 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:21.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:21.344 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:21.341+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:21.444 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:21.442+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:21.491 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:21.491 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:21.491 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:21.491 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:21.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:21.545 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:21.543+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:21.601 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-17T21:46:21.606 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:21.607 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:21.607 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:21.607 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:21.607 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:21.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:21.657 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:21.654+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:21.687 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:46:21.747 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:21.744+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:21.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8184326769999997, 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:21.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8164076250000001, 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:21.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8024215689999998, 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8004603619999999, 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7984270049999997, 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7964490870000001, 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7964262929999997, 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11260 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 831488~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975077+0000", 2024-09-17T21:46:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.794513147, 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065155878, 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11260 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975077+0000", 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975077+0000", 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.976946+0000", 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001869445 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.976950+0000", 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5039999999999998e-06 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.976962+0000", 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2887e-05 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.976970+0000", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2359999999999998e-06 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040076+0000", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063106428000000006 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040125+0000", 2024-09-17T21:46:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060999999999997e-05 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040232+0000", 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000107317 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11276 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1617920~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979061+0000", 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7905289680000003, 2024-09-17T21:46:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065216000999999996, 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11276 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979061+0000", 2024-09-17T21:46:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979061+0000", 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979061+0000", 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979062+0000", 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.221e-06 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979066+0000", 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979068+0000", 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044127+0000", 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065058853999999999 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044175+0000", 2024-09-17T21:46:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8063999999999999e-05 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044277+0000", 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010139199999999999 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7905172299999998, 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7884861709999997, 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7884595970000001, 2024-09-17T21:46:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7878498040000004, 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7878322180000001, 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7878172609999998, 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0721357730000003, 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0721051519999998, 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0720805340000004, 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0720585869999999, 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0720371169999998, 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:21.879 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:21.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:21.881 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:21.878+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:21.981 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:21.978+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:22.028 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:22.028 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:22.028 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:22.028 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:22.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:22.082 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:22.080+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:22.183 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:22.180+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:22.184 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:22.184 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:22.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:22.284 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:22.282+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3667406350000002, 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3667181419999999, 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3666938819999999, 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3666679989999997, 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3666429979999997, 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:22.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.366603199, 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:22.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.370 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3665874540000003, 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:22.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:22.375 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3665759189999998, 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:22.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3665656190000002, 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.380 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:22.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:22.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645943760000003, 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:22.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:22.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645812279999996, 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:22.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:22.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645706220000005, 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:22.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:22.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645602840000004, 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:22.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645502770000002, 2024-09-17T21:46:22.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:22.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:22.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645403260000002, 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3645250180000001, 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:22.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3625491309999997, 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:22.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:22.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3625381750000001, 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3625256859999997, 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:22.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.3625152229999999, 2024-09-17T21:46:22.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:22.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:22.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:22.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:22.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:22.436 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:22.437 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:22.435+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:22.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:22.538 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:22.535+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3.bc9c1801 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.87351568199999996, 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.87353965099999997, 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:22.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:22.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:22.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:22.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:22.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3.f18d355b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84124070699999998, 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84131296499999997, 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.563 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:22.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:22.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3.f18d355b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84100193899999998, 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.84110862799999997, 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:22.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:22.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:22.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 3 2024-09-17T21:46:22.571 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:22.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:22.639 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:22.636+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:22.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:22.680 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:22.680 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:22.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:22.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:22.777 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:22.775+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:22.809 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:22.809 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:22.830 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:22.830 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:22.840 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:22.838+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:22.848 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:22.849 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:22.853 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:22.853 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:22.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:22.866 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:22.866 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:22.887 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:22.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:22.904 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:22.905 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:22.956 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:22.954+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:22.979 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:22.980 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:22.989 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:23.060 INFO:teuthology.orchestra.run.smithi165.stdout:819 trunc from 0xe0bc200 to 0x44dec00 2024-09-17T21:46:23.073 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:23.071+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:23.143 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:23.141+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:23.161 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:23.161 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:23.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:23.186 INFO:teuthology.orchestra.run.smithi165.stdout:820 write 0x8735e00 thru 0x8736eda (0x10db bytes) 2024-09-17T21:46:23.186 INFO:teuthology.orchestra.run.smithi165.stdout:821 read 0x19a6a00 thru 0x19b6998 (0xff99 bytes) 2024-09-17T21:46:23.186 INFO:teuthology.orchestra.run.smithi165.stdout:824 punch from 0x12fe800 to 0x130c000, (0xd800 bytes) 2024-09-17T21:46:23.244 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:23.242+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:23.290 INFO:teuthology.orchestra.run.smithi165.stdout:825 write 0xe6f3400 thru 0xe6f6b78 (0x3779 bytes) 2024-09-17T21:46:23.297 INFO:teuthology.orchestra.run.smithi165.stdout:827 read 0x2c46a00 thru 0x2c52ed0 (0xc4d1 bytes) 2024-09-17T21:46:23.298 INFO:teuthology.orchestra.run.smithi165.stdout:830 write 0xa783800 thru 0xa78f48b (0xbc8c bytes) 2024-09-17T21:46:23.308 INFO:teuthology.orchestra.run.smithi165.stdout:833 trunc from 0xe6f6c00 to 0x5e66400 2024-09-17T21:46:23.329 INFO:teuthology.orchestra.run.smithi165.stdout:834 clone 7 order 18 su 8192 sc 12 2024-09-17T21:46:23.345 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:23.343+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:23.349 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:23.349 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:23.359 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:23.446 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:23.444+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:23.506 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:23.506 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:23.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:23.547 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:23.544+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:23.647 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:23.645+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:23.662 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:23.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:23.693 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:23.693 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:23.693 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [1, 2, 3, 4, 5, 0] 2024-09-17T21:46:23.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-17T21:46:23.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:46:23.748 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:23.745+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:23.767 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone6 from 0x5e66400 (overlap 0x44dec00) to 0x3718800 2024-09-17T21:46:23.848 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:23.846+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:23.873 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:23.873 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:23.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:23.953 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:23.950+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:24.051 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:24.048+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:24.070 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.071 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:24.151 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:24.149+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:24.208 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.208 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:24.255 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:24.252+0000 7f0c28614640 -1 mon.a@0(leader).osd e41 definitely_dead 0 2024-09-17T21:46:24.258 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:24.256+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:24.353 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:24.351+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:24.365 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.365 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.373 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:24.455 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:24.452+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:24.503 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-17T21:46:24.529 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.530 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:24.555 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:24.553+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:24.656 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:24.654+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:24.688 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.688 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:24.704 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.0. 2024-09-17T21:46:24.704 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:46:24.762 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:24.760+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:24.848 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.848 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:24.858 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:24.856+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:24.916 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:24.959 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:24.957+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:24.963 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:24.963 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:24.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:25.040 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:25.040 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:25.057 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:25.057 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:25.060 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:25.058+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:25.073 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:25.074 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:25.090 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:25.107 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:25.107 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:25.120 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:25.120 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:25.123 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:25.124 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:25.129 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:25.173 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:25.171+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:25.237 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2635285730000003, 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:25.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:25.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2635060799999991, 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:25.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:25.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2634818199999991, 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:25.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2634559369999998, 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:25.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:25.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2634309360000007, 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:25.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:25.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2633911369999993, 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:25.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:25.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:25.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2633753920000004, 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:25.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2633638569999999, 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:25.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:25.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2633535570000003, 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:25.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613823140000004, 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:25.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:25.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613691659999997, 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:25.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613585599999997, 2024-09-17T21:46:25.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:25.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613482220000005, 2024-09-17T21:46:25.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:25.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613382150000003, 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:25.309 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613282639999994, 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:25.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2613129559999994, 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:25.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:25.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2593370690000008, 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:25.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2593261130000002, 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:25.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2593136240000007, 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:25.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:25.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2593031610000001, 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:25.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:25.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:25.342 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:25.344 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:25.345 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:25.341+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:25.445 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:25.443+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:25.453 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:25.453 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:25.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:25.546 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:25.544+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:25.608 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:25.609 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:25.617 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:25.658 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:25.656+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:25.748 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:25.745+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:25.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:25.793 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:25.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.507162595, 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:25.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:25.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:25.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:25.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.485235791000001, 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:25.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:25.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.133847755, 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:25.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:25.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:25.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.628918637, 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:25.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.628908900999999, 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:25.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.62889929, 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:25.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:25.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.628888459000001, 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:25.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.628874868, 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:25.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:25.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.628864518, 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:25.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:25.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.623752141000001, 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:25.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.619425445999999, 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:25.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.619286237000001, 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:25.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.619269247, 2024-09-17T21:46:25.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.619248249, 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.614402094999999, 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.614389685999999, 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.614379354, 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1071354580000001, 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0748604830000001, 2024-09-17T21:46:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.0746217150000001, 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:25.894 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:25.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:25.896 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:25.893+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:25.997 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:25.994+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:26.003 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:26.003 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:26.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:26.098 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:26.095+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:26.160 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:26.161 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:26.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:26.199 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:26.196+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:26.300 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:26.298+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:26.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4111650139999998, 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:26.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4111547130000002, 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:26.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:26.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4111386390000007, 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4111274260000002, 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:26.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:26.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4111099879999998, 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:26.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4110991179999992, 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:26.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4109042410000008, 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:26.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:26.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4091995710000003, 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:26.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4091889329999994, 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:26.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:26.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:26.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4091788399999992, 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:26.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:26.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4091687210000003, 2024-09-17T21:46:26.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:26.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.377 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4070384909999998, 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.378 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:26.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4070287599999993, 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4062454009999996, 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:26.388 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:26.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:26.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:26.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4062298309999992, 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:26.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:26.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4061369290000005, 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:26.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4056423099999993, 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:26.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4054982650000003, 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:26.410 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:26.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4048128280000007, 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:26.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.4044504379999996, 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:26.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.426 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:26.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:26.428 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:26.425+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:26.528 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:26.526+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:26.576 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.576 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:26.576 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:26.576 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:26.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:26.628 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:26.626+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:26.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:46:26.741 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:26.742 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:26.742 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:26.739+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:26.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:26.839 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:26.837+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:26.931 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:26.928+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:26.944 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:26.944 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4542.0:1218 3.9 e30/21 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head v 30'67, mlcod=29'64)", 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.165965196, 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067374460000000002, 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:26.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775156+0000", 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775164+0000", 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775185+0000", 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0710999999999999e-05 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775197+0000", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1739000000000001e-05 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775269+0000", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775290+0000", 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0866999999999999e-05 2024-09-17T21:46:26.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781826+0000", 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065362390000000001 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781893+0000", 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6446999999999996e-05 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.131196433, 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:26.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:26.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11451 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9387129089999995, 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053110780000000003, 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11451 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002409+0000", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002413+0000", 2024-09-17T21:46:26.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4029999999999999e-06 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002415+0000", 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007559+0000", 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005143173 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007608+0000", 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9521999999999998e-05 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007719+0000", 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011066299999999999 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11452 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:26.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9387014170000008, 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054903180000000001, 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11452 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:26.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002424+0000", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002431+0000", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.905e-06 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002434+0000", 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.083e-06 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007757+0000", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053229310000000004 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007805+0000", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8090000000000002e-05 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007909+0000", 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104514 2024-09-17T21:46:26.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11453 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1515520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386828979999997, 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056581649999999997, 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11453 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:26.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4400000000000001e-07 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002439+0000", 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3740000000000001e-06 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002443+0000", 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9519999999999996e-06 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002448+0000", 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9239999999999998e-06 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007946+0000", 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054980029999999996 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007992+0000", 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008096+0000", 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103478 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11454 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1564672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386689990000008, 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058292750000000001, 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11454 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:26.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002453+0000", 2024-09-17T21:46:26.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.28e-06 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002457+0000", 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3529999999999997e-06 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002460+0000", 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008133+0000", 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056725229999999996 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008178+0000", 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5463999999999997e-05 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008281+0000", 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102674 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11455 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386571989999997, 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060027099999999996, 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11455 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.974 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002464+0000", 2024-09-17T21:46:26.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-07 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002465+0000", 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.234e-06 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002469+0000", 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0759999999999996e-06 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002472+0000", 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008318+0000", 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058459009999999997 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008364+0000", 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5948999999999997e-05 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008466+0000", 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010257200000000001 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11457 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386457719999992, 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061758150000000003, 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11457 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002476+0000", 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2109999999999999e-06 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002480+0000", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002484+0000", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.253e-06 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008503+0000", 2024-09-17T21:46:26.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060196650000000004 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008548+0000", 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5223999999999999e-05 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008651+0000", 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102143 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11458 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386321550000005, 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063602019999999997, 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11458 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002489+0000", 2024-09-17T21:46:26.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002491+0000", 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.968e-06 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002498+0000", 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002503+0000", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.254e-06 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008688+0000", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061850450000000001 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008745+0000", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6647999999999997e-05 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008849+0000", 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010412900000000001 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11459 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9386108130000004, 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065278610000000003, 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11459 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:26.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002513+0000", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3590000000000002e-06 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002525+0000", 2024-09-17T21:46:26.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2245e-05 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002531+0000", 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008887+0000", 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063556790000000004 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008934+0000", 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7157000000000003e-05 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009038+0000", 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103892 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11460 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9385819719999997, 2024-09-17T21:46:26.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066803280000000001, 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11460 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002542+0000", 2024-09-17T21:46:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002551+0000", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6229999999999992e-06 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002557+0000", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009068+0000", 2024-09-17T21:46:26.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065103599999999998 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009115+0000", 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7818999999999997e-05 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009219+0000", 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010344999999999999 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:26.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11461 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1859584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9385505149999993, 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068348430000000002, 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11461 2024-09-17T21:46:26.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:26.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002571+0000", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002573+0000", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002582+0000", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2809999999999997e-06 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:26.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002588+0000", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9819999999999996e-06 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009256+0000", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066681739999999998 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009303+0000", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6663999999999999e-05 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:26.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009405+0000", 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101753 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11462 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1908736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9385177149999997, 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006995823, 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:27.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11462 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:27.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002606+0000", 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.807e-06 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002614+0000", 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8259999999999993e-06 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002620+0000", 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8869999999999997e-06 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009442+0000", 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068217820000000002 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009497+0000", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5600000000000003e-05 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009599+0000", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101369 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11463 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1957888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:27.004 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9384929870000001, 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071654559999999997, 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11463 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002631+0000", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002638+0000", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9909999999999996e-06 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002643+0000", 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009636+0000", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069926459999999999 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009682+0000", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6406000000000003e-05 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009793+0000", 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011097799999999999 2024-09-17T21:46:27.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11464 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.93847083, 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073216799999999997, 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.009 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11464 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002652+0000", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002659+0000", 2024-09-17T21:46:27.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8569999999999997e-06 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002665+0000", 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.789e-06 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009824+0000", 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071584190000000001 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009871+0000", 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.782e-05 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009971+0000", 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100076 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11465 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9384465750000004, 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074631050000000003, 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11465 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002681+0000", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002690+0000", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5639999999999997e-06 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002711+0000", 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009992+0000", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072811359999999997 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010037+0000", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4721000000000001e-05 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010137+0000", 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100447 2024-09-17T21:46:27.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15737 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'905, mlcod=35'900)", 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9135436549999998, 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052510500000000002, 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027583+0000", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.716e-06 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027601+0000", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7606000000000001e-05 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027610+0000", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027668+0000", 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8301999999999998e-05 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027687+0000", 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8567e-05 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032787+0000", 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050997359999999997 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032828+0000", 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1248999999999998e-05 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15743 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'911, mlcod=35'903)", 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9084505400000005, 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052577359999999998, 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032671+0000", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899999999999995e-07 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032676+0000", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032686+0000", 2024-09-17T21:46:27.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-05 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032693+0000", 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032952+0000", 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025868400000000002 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032976+0000", 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3995e-05 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037203+0000", 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042275050000000003 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037928+0000", 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000724411 2024-09-17T21:46:27.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15901 3.18 3:18be054b:::rbd_data.11121f8e87bb.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8876379940000003, 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054083960000000002, 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15901 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9400000000000005e-07 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053488+0000", 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2910000000000001e-06 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053497+0000", 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6800000000000005e-06 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053504+0000", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053558+0000", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3987000000000003e-05 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053679+0000", 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120191 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053977+0000", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029810399999999999 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.056472+0000", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024953789999999998 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058804+0000", 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002332373 2024-09-17T21:46:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058855+0000", 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.024e-05 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058891+0000", 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6359000000000002e-05 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15903 3.7 3:e5ee49b6:::rbd_data.11121f8e87bb.0000000000000067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8874309399999998, 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055705099999999999, 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15903 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:27.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6499999999999999e-07 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053699+0000", 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054218+0000", 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051917300000000005 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054227+0000", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0270000000000001e-06 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054307+0000", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0543000000000005e-05 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054381+0000", 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3633999999999993e-05 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054482+0000", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010119300000000001 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.055778+0000", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012961120000000001 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059175+0000", 2024-09-17T21:46:27.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033961880000000001 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059226+0000", 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1353000000000001e-05 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059260+0000", 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4289000000000003e-05 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:27.038 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:27.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:27.040 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:27.037+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:27.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:27.140 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:27.138+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:27.148 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:27.148 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:27.149 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:27.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:27.244 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:27.242+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:27.273 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:27.290 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:27.290 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:27.306 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:27.306 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:27.317 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:27.317 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:27.323 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:27.323 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:27.326 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:27.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:27.340 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:27.357 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:27.357 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:27.367 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:27.364+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:27.443 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:27.441+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:27.491 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:27.491 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:27.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:27.547 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:27.545+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:27.645 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:27.643+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:27.667 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:27.667 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:27.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:27.713 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:46:27.764 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:27.762+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:27.828 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:27.828 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:27.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:27.847 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:27.844+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:27.947 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:27.945+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:27.983 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:27.984 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:27.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:28.048 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:28.046+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:28.099 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.099 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:28.150 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:28.147+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:28.214 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.214 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:28.266 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:28.264+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:28.330 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.330 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.338 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:28.380 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:28.377+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:28.453 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:28.450+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:28.455 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.455 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:28.554 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:28.551+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:28.571 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.571 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:28.655 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:28.652+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:28.687 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.687 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.696 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:28.755 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:28.753+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:28.827 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.827 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:28.857 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:28.854+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:28.949 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:28.949 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:28.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:28.958 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:28.955+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:29.058 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:29.056+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:29.074 INFO:teuthology.orchestra.run.smithi165.stdout:835 write 0x3baae00 thru 0x3bb0b1c (0x5d1d bytes) 2024-09-17T21:46:29.075 INFO:teuthology.orchestra.run.smithi165.stdout:839 read 0x55ca800 thru 0x55cf67f (0x4e80 bytes) 2024-09-17T21:46:29.076 INFO:teuthology.orchestra.run.smithi165.stdout:842 punch from 0x99a400 to 0x9a1200, (0x6e00 bytes) 2024-09-17T21:46:29.101 INFO:teuthology.orchestra.run.smithi165.stdout:843 read 0x531de00 thru 0x5323e27 (0x6028 bytes) 2024-09-17T21:46:29.106 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:29.106 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:29.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:29.124 INFO:teuthology.orchestra.run.smithi165.stdout:848 write 0xce5cc00 thru 0xce5e9d4 (0x1dd5 bytes) 2024-09-17T21:46:29.138 INFO:teuthology.orchestra.run.smithi165.stdout:851 trunc from 0xce5ea00 to 0x1be3600 2024-09-17T21:46:29.160 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:29.158+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:29.191 INFO:teuthology.orchestra.run.smithi165.stdout:852 read 0x45f800 thru 0x46def6 (0xe6f7 bytes) 2024-09-17T21:46:29.191 INFO:teuthology.orchestra.run.smithi165.stdout:853 read 0xfac400 thru 0xfadddf (0x19e0 bytes) 2024-09-17T21:46:29.192 INFO:teuthology.orchestra.run.smithi165.stdout:855 read 0xead600 thru 0xeba882 (0xd283 bytes) 2024-09-17T21:46:29.194 INFO:teuthology.orchestra.run.smithi165.stdout:858 trunc from 0x1be3600 to 0x2f1a600 2024-09-17T21:46:29.214 INFO:teuthology.orchestra.run.smithi165.stdout:863 trunc from 0x2f1a600 to 0x73dd600 2024-09-17T21:46:29.228 INFO:teuthology.orchestra.run.smithi165.stdout:865 read 0x385c600 thru 0x38648e9 (0x82ea bytes) 2024-09-17T21:46:29.229 INFO:teuthology.orchestra.run.smithi165.stdout:866 trunc from 0x73dd600 to 0x84ba000 2024-09-17T21:46:29.230 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:29.230 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:29.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:29.244 INFO:teuthology.orchestra.run.smithi165.stdout:867 punch from 0x5ca400 to 0x5cc200, (0x1e00 bytes) 2024-09-17T21:46:29.245 INFO:teuthology.orchestra.run.smithi165.stdout:868 write 0x3a60000 thru 0x3a62afe (0x2aff bytes) 2024-09-17T21:46:29.245 INFO:teuthology.orchestra.run.smithi165.stdout:869 read 0x898400 thru 0x89e807 (0x6408 bytes) 2024-09-17T21:46:29.251 INFO:teuthology.orchestra.run.smithi165.stdout:870 trunc from 0x84ba000 to 0xad4b800 2024-09-17T21:46:29.260 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:29.258+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:29.271 INFO:teuthology.orchestra.run.smithi165.stdout:871 read 0x956a400 thru 0x9574f1b (0xab1c bytes) 2024-09-17T21:46:29.271 INFO:teuthology.orchestra.run.smithi165.stdout:872 punch from 0x87df600 to 0x87ee800, (0xf200 bytes) 2024-09-17T21:46:29.281 INFO:teuthology.orchestra.run.smithi165.stdout:874 trunc from 0xad4b800 to 0x2a3e800 2024-09-17T21:46:29.305 INFO:teuthology.orchestra.run.smithi165.stdout:875 punch from 0x24a7000 to 0x24b0200, (0x9200 bytes) 2024-09-17T21:46:29.305 INFO:teuthology.orchestra.run.smithi165.stdout:876 read 0x246c200 thru 0x2476b5a (0xa95b bytes) 2024-09-17T21:46:29.305 INFO:teuthology.orchestra.run.smithi165.stdout:878 punch from 0x3d0200 to 0x3d2400, (0x2200 bytes) 2024-09-17T21:46:29.333 INFO:teuthology.orchestra.run.smithi165.stdout:880 write 0xc3e1000 thru 0xc3e3949 (0x294a bytes) 2024-09-17T21:46:29.333 INFO:teuthology.orchestra.run.smithi165.stdout:883 read 0x8873800 thru 0x8875f1d (0x271e bytes) 2024-09-17T21:46:29.333 INFO:teuthology.orchestra.run.smithi165.stdout:884 trunc from 0xc3e3a00 to 0xbe0aa00 2024-09-17T21:46:29.361 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:29.359+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:29.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:29.373 INFO:teuthology.orchestra.run.smithi165.stdout:886 write 0xcf3fc00 thru 0xcf48df0 (0x91f1 bytes) 2024-09-17T21:46:29.373 INFO:teuthology.orchestra.run.smithi165.stdout:887 write 0x4f17a00 thru 0x4f1c650 (0x4c51 bytes) 2024-09-17T21:46:29.374 INFO:teuthology.orchestra.run.smithi165.stdout:888 write 0x335e00 thru 0x33aae0 (0x4ce1 bytes) 2024-09-17T21:46:29.380 INFO:teuthology.orchestra.run.smithi165.stdout:889 trunc from 0xcf48e00 to 0x8305000 2024-09-17T21:46:29.395 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:29.395 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:29.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:29.422 INFO:teuthology.orchestra.run.smithi165.stdout:891 read 0x71c7a00 thru 0x71d6456 (0xea57 bytes) 2024-09-17T21:46:29.422 INFO:teuthology.orchestra.run.smithi165.stdout:892 write 0x54dca00 thru 0x54dee1e (0x241f bytes) 2024-09-17T21:46:29.422 INFO:teuthology.orchestra.run.smithi165.stdout:896 read 0x8089800 thru 0x808cc35 (0x3436 bytes) 2024-09-17T21:46:29.422 INFO:teuthology.orchestra.run.smithi165.stdout:897 punch from 0x72b4a00 to 0x72be400, (0x9a00 bytes) 2024-09-17T21:46:29.423 INFO:teuthology.orchestra.run.smithi165.stdout:898 read 0x5977200 thru 0x5984bab (0xd9ac bytes) 2024-09-17T21:46:29.423 INFO:teuthology.orchestra.run.smithi165.stdout:899 write 0x1640200 thru 0x164b443 (0xb244 bytes) 2024-09-17T21:46:29.423 INFO:teuthology.orchestra.run.smithi165.stdout:900 punch from 0x550f800 to 0x551ce00, (0xd600 bytes) 2024-09-17T21:46:29.425 INFO:teuthology.orchestra.run.smithi165.stdout:901 read 0x5131a00 thru 0x513e6a6 (0xcca7 bytes) 2024-09-17T21:46:29.425 INFO:teuthology.orchestra.run.smithi165.stdout:902 read 0x30d4800 thru 0x30dbf2a (0x772b bytes) 2024-09-17T21:46:29.425 INFO:teuthology.orchestra.run.smithi165.stdout:903 read 0x6110800 thru 0x611bc08 (0xb409 bytes) 2024-09-17T21:46:29.426 INFO:teuthology.orchestra.run.smithi165.stdout:904 read 0x60dee00 thru 0x60e70f3 (0x82f4 bytes) 2024-09-17T21:46:29.433 INFO:teuthology.orchestra.run.smithi165.stdout:905 trunc from 0x8305000 to 0xe2a4200 2024-09-17T21:46:29.470 INFO:teuthology.orchestra.run.smithi165.stdout:913 punch from 0x41dbe00 to 0x41dc000, (0x200 bytes) 2024-09-17T21:46:29.470 INFO:teuthology.orchestra.run.smithi165.stdout:914 read 0xd14b600 thru 0xd14c144 (0xb45 bytes) 2024-09-17T21:46:29.470 INFO:teuthology.orchestra.run.smithi165.stdout:915 read 0xd08200 thru 0xd16613 (0xe414 bytes) 2024-09-17T21:46:29.471 INFO:teuthology.orchestra.run.smithi165.stdout:916 write 0xb1f1600 thru 0xb1fdfab (0xc9ac bytes) 2024-09-17T21:46:29.471 INFO:teuthology.orchestra.run.smithi165.stdout:921 write 0x3b54000 thru 0x3b57c1f (0x3c20 bytes) 2024-09-17T21:46:29.476 INFO:teuthology.orchestra.run.smithi165.stdout:922 trunc from 0xe2a4200 to 0x9ac9200 2024-09-17T21:46:29.489 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:29.487+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:29.504 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:29.504 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:29.509 INFO:teuthology.orchestra.run.smithi165.stdout:926 write 0x40e5c00 thru 0x40f0394 (0xa795 bytes) 2024-09-17T21:46:29.510 INFO:teuthology.orchestra.run.smithi165.stdout:929 trunc from 0x9ac9200 to 0xa198c00 2024-09-17T21:46:29.538 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:29.539 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:29.542 INFO:teuthology.orchestra.run.smithi165.stdout:931 trunc from 0xa198c00 to 0x17c8c00 2024-09-17T21:46:29.560 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:29.563 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:29.561+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:29.579 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4542.0:1218 3.9 e30/21 3:939f92a8:::rbd_data.11125c054388.0000000000000003:head v 30'67, mlcod=29'64)", 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.800471743999999, 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067374460000000002, 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775155+0000", 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775156+0000", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3500000000000002e-07 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775164+0000", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6889999999999993e-06 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775185+0000", 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0710999999999999e-05 2024-09-17T21:46:29.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775197+0000", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1739000000000001e-05 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775269+0000", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2418999999999999e-05 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.775290+0000", 2024-09-17T21:46:29.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0866999999999999e-05 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781826+0000", 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065362390000000001 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.781893+0000", 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6446999999999996e-05 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.765702981, 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:29.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:29.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:29.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:29.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11451 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1417216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573219457, 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053110780000000003, 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11451 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002408+0000", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5499999999999999e-07 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002409+0000", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002413+0000", 2024-09-17T21:46:29.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4029999999999999e-06 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002415+0000", 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007559+0000", 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005143173 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007608+0000", 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9521999999999998e-05 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007719+0000", 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011066299999999999 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11452 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1466368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573207965, 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054903180000000001, 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11452 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002419+0000", 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002424+0000", 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5800000000000002e-06 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002431+0000", 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.905e-06 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002434+0000", 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.083e-06 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007757+0000", 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053229310000000004 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007805+0000", 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8090000000000002e-05 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007909+0000", 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104514 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11453 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1515520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573189446000001, 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056581649999999997, 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11453 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002438+0000", 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4400000000000001e-07 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002439+0000", 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3740000000000001e-06 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002443+0000", 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9519999999999996e-06 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002448+0000", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9239999999999998e-06 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007946+0000", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054980029999999996 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.007992+0000", 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6190000000000003e-05 2024-09-17T21:46:29.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008096+0000", 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103478 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11454 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1564672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573175547, 2024-09-17T21:46:29.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058292750000000001, 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11454 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:29.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002452+0000", 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2100000000000001e-07 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002453+0000", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.28e-06 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002457+0000", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3529999999999997e-06 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002460+0000", 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:46:29.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008133+0000", 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056725229999999996 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008178+0000", 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5463999999999997e-05 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008281+0000", 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102674 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11455 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573163747000001, 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060027099999999996, 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11455 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002463+0000", 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002464+0000", 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.17e-07 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002465+0000", 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.234e-06 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002469+0000", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0759999999999996e-06 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002472+0000", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008318+0000", 2024-09-17T21:46:29.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058459009999999997 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008364+0000", 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5948999999999997e-05 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008466+0000", 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010257200000000001 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11457 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.57315232, 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061758150000000003, 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11457 2024-09-17T21:46:29.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002475+0000", 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002476+0000", 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2109999999999999e-06 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002480+0000", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002484+0000", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.253e-06 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008503+0000", 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060196650000000004 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008548+0000", 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5223999999999999e-05 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008651+0000", 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102143 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11458 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573138703, 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063602019999999997, 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11458 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002488+0000", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002489+0000", 2024-09-17T21:46:29.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002491+0000", 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.968e-06 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002498+0000", 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8240000000000002e-06 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002503+0000", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.254e-06 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008688+0000", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0061850450000000001 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008745+0000", 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6647999999999997e-05 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008849+0000", 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010412900000000001 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11459 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573117361, 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065278610000000003, 2024-09-17T21:46:29.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11459 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002510+0000", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002513+0000", 2024-09-17T21:46:29.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3590000000000002e-06 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002525+0000", 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2245e-05 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002531+0000", 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008887+0000", 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063556790000000004 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.008934+0000", 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7157000000000003e-05 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009038+0000", 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103892 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11460 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573088520000001, 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066803280000000001, 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11460 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002539+0000", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002542+0000", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4940000000000002e-06 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002551+0000", 2024-09-17T21:46:29.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6229999999999992e-06 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002557+0000", 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1489999999999999e-06 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009068+0000", 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065103599999999998 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009115+0000", 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7818999999999997e-05 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009219+0000", 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010344999999999999 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11461 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1859584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573057063, 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068348430000000002, 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11461 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002570+0000", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002571+0000", 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:46:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002573+0000", 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002582+0000", 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2809999999999997e-06 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002588+0000", 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9819999999999996e-06 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009256+0000", 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066681739999999998 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009303+0000", 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6663999999999999e-05 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009405+0000", 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101753 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11462 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1908736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.573024263000001, 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.006995823, 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11462 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002603+0000", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5199999999999997e-07 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002606+0000", 2024-09-17T21:46:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.807e-06 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002614+0000", 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8259999999999993e-06 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002620+0000", 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8869999999999997e-06 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009442+0000", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0068217820000000002 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009497+0000", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5600000000000003e-05 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009599+0000", 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101369 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11463 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 1957888~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.572999534999999, 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071654559999999997, 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11463 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002628+0000", 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002631+0000", 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002638+0000", 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9909999999999996e-06 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002643+0000", 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4929999999999998e-06 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009636+0000", 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069926459999999999 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009682+0000", 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6406000000000003e-05 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009793+0000", 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011097799999999999 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11464 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.572977378000001, 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073216799999999997, 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11464 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002650+0000", 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002652+0000", 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3010000000000001e-06 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002659+0000", 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8569999999999997e-06 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002665+0000", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.789e-06 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009824+0000", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071584190000000001 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009871+0000", 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.782e-05 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009971+0000", 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100076 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11465 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.572953123, 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074631050000000003, 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11465 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002681+0000", 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:46:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002690+0000", 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5639999999999997e-06 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002711+0000", 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009992+0000", 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072811359999999997 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010037+0000", 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4721000000000001e-05 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010137+0000", 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100447 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15737 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'905, mlcod=35'900)", 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.548050203000001, 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052510500000000002, 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027577+0000", 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027583+0000", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.716e-06 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027601+0000", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7606000000000001e-05 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027610+0000", 2024-09-17T21:46:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027668+0000", 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8301999999999998e-05 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.027687+0000", 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8567e-05 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032787+0000", 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050997359999999997 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032828+0000", 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1248999999999998e-05 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4481.0:15743 3.1 e35/34 3:845283c9:::rbd_object_map.11121f8e87bb:head v 35'911, mlcod=35'903)", 2024-09-17T21:46:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.542957088, 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052577359999999998, 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032670+0000", 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032671+0000", 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899999999999995e-07 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032676+0000", 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4029999999999998e-06 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032686+0000", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-05 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032693+0000", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6889999999999998e-06 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032952+0000", 2024-09-17T21:46:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00025868400000000002 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.032976+0000", 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3995e-05 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037203+0000", 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0042275050000000003 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.037928+0000", 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000724411 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15901 3.18 3:18be054b:::rbd_data.11121f8e87bb.0000000000000066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.522144541999999, 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0054083960000000002, 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15901 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053483+0000", 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9400000000000005e-07 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053488+0000", 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2910000000000001e-06 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053497+0000", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6800000000000005e-06 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053504+0000", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0979999999999996e-06 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053558+0000", 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3987000000000003e-05 2024-09-17T21:46:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053679+0000", 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120191 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053977+0000", 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029810399999999999 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.056472+0000", 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024953789999999998 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058804+0000", 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002332373 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058855+0000", 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.024e-05 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.058891+0000", 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6359000000000002e-05 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:15903 3.7 3:e5ee49b6:::rbd_data.11121f8e87bb.0000000000000067:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.521937488000001, 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0055705099999999999, 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15903 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053690+0000", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6499999999999999e-07 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.053699+0000", 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3329999999999997e-06 2024-09-17T21:46:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054218+0000", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051917300000000005 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054227+0000", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0270000000000001e-06 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054307+0000", 2024-09-17T21:46:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0543000000000005e-05 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054381+0000", 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3633999999999993e-05 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.054482+0000", 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010119300000000001 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.055778+0000", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012961120000000001 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059175+0000", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033961880000000001 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059226+0000", 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1353000000000001e-05 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:18.059260+0000", 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4289000000000003e-05 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:29.678 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:29.679 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:29.680 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:29.677+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:29.680 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:29.680 INFO:teuthology.orchestra.run.smithi165.stdout:932 punch from 0x34000 to 0x3ae00, (0x6e00 bytes) 2024-09-17T21:46:29.680 INFO:teuthology.orchestra.run.smithi165.stdout:933 write 0xd759600 thru 0xd7677f3 (0xe1f4 bytes) 2024-09-17T21:46:29.681 INFO:teuthology.orchestra.run.smithi165.stdout:934 trunc from 0xd767800 to 0x703c400 2024-09-17T21:46:29.681 INFO:teuthology.orchestra.run.smithi165.stdout:935 read 0xf11a00 thru 0xf16a73 (0x5074 bytes) 2024-09-17T21:46:29.681 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:29.682 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:29.682 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:29.682 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:29.683 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:29.685 INFO:teuthology.orchestra.run.smithi165.stdout:939 write 0xbf01400 thru 0xbf0889e (0x749f bytes) 2024-09-17T21:46:29.686 INFO:teuthology.orchestra.run.smithi165.stdout:942 trunc from 0xbf08a00 to 0xbc6d400 2024-09-17T21:46:29.722 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:29.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:46:29.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:46:29.731 INFO:teuthology.orchestra.run.smithi165.stdout:944 write 0xce84600 thru 0xce89eed (0x58ee bytes) 2024-09-17T21:46:29.744 INFO:teuthology.orchestra.run.smithi165.stdout:947 trunc from 0xce8a000 to 0x2b1e600 2024-09-17T21:46:29.770 INFO:teuthology.orchestra.run.smithi165.stdout:949 read 0x1e35400 thru 0x1e3a996 (0x5597 bytes) 2024-09-17T21:46:29.771 INFO:teuthology.orchestra.run.smithi165.stdout:952 write 0x2fa600 thru 0x2ff0a8 (0x4aa9 bytes) 2024-09-17T21:46:29.772 INFO:teuthology.orchestra.run.smithi165.stdout:954 read 0x1d36c00 thru 0x1d40bf5 (0x9ff6 bytes) 2024-09-17T21:46:29.772 INFO:teuthology.orchestra.run.smithi165.stdout:957 write 0x14f0200 thru 0x14f29d8 (0x27d9 bytes) 2024-09-17T21:46:29.773 INFO:teuthology.orchestra.run.smithi165.stdout:959 read 0x75b000 thru 0x76060c (0x560d bytes) 2024-09-17T21:46:29.774 INFO:teuthology.orchestra.run.smithi165.stdout:961 punch from 0x29a0600 to 0x29ae800, (0xe200 bytes) 2024-09-17T21:46:29.775 INFO:teuthology.orchestra.run.smithi165.stdout:962 read 0x1ea1400 thru 0x1eb06c1 (0xf2c2 bytes) 2024-09-17T21:46:29.775 INFO:teuthology.orchestra.run.smithi165.stdout:963 clone 8 order 20 su 16384 sc 14 2024-09-17T21:46:29.780 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:29.778+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:29.815 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:29.815 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:29.816 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:29.816 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:29.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:29.909 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:29.906+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:29.972 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:29.973 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:29.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:30.022 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:30.019+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:30.084 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:30.082+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.967824360999998, 2024-09-17T21:46:30.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:30.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:30.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11508 3.11 3:883b5b9d:::rbd_data.11125c054388.0000000000000008:7 [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:30.137 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.123245988000001, 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011205596, 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11508 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003380+0000", 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2600000000000001e-07 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003385+0000", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003388+0000", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5590000000000001e-06 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.003391+0000", 2024-09-17T21:46:30.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014342+0000", 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010951106 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014423+0000", 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1283000000000001e-05 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.014585+0000", 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016200700000000001 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:2636 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [watch reconnect cookie 140714606129008 gen 5] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.769668963999999, 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011916717, 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2636 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356957+0000", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.63e-07 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356964+0000", 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4880000000000004e-06 2024-09-17T21:46:30.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356969+0000", 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3800000000000002e-06 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.356972+0000", 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0929999999999999e-06 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367098+0000", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010125861 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367173+0000", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4714000000000003e-05 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.368874+0000", 2024-09-17T21:46:30.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017007179999999999 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:12952 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.769612012, 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011082310999999999, 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 12952 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357014+0000", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357015+0000", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357019+0000", 2024-09-17T21:46:30.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6410000000000001e-06 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357026+0000", 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4280000000000001e-06 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.357030+0000", 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3150000000000002e-06 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367922+0000", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010892208 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.367999+0000", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7173000000000001e-05 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.368096+0000", 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6699000000000004e-05 2024-09-17T21:46:30.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13672 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671385088] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607382680000001, 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011128726, 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13672 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519243+0000", 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519244+0000", 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519245+0000", 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519249+0000", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4460000000000001e-06 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519252+0000", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.061e-06 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530172+0000", 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010919836 2024-09-17T21:46:30.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530238+0000", 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6672000000000001e-05 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530372+0000", 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000133511 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13673 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633670968352] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607368878999999, 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011284531, 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13673 2024-09-17T21:46:30.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519257+0000", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.39e-07 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519259+0000", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.508e-06 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519262+0000", 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1540000000000002e-06 2024-09-17T21:46:30.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519265+0000", 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5950000000000001e-06 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530412+0000", 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011146937000000001 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530456+0000", 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4070000000000003e-05 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530542+0000", 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5927999999999995e-05 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13674 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671395568] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607339284, 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011500078, 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13674 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519287+0000", 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:30.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519289+0000", 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7320000000000001e-06 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519292+0000", 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3139999999999999e-06 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519295+0000", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6759999999999999e-06 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530580+0000", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011285117000000001 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530654+0000", 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4134999999999994e-05 2024-09-17T21:46:30.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530787+0000", 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013265400000000001 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13675 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671407856] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607327771, 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011670329, 2024-09-17T21:46:30.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13675 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519298+0000", 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519300+0000", 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.404e-06 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519303+0000", 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519306+0000", 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6579999999999998e-06 2024-09-17T21:46:30.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530831+0000", 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011525844 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530876+0000", 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4885999999999997e-05 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.530968+0000", 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2119000000000004e-05 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13676 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671128672] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607307892, 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011888612, 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13676 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519318+0000", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1300000000000001e-07 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519320+0000", 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.857e-06 2024-09-17T21:46:30.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519325+0000", 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5519999999999998e-06 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519328+0000", 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.88e-06 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531010+0000", 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011682203 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531076+0000", 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5748999999999997e-05 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531207+0000", 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000130958 2024-09-17T21:46:30.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13677 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671428000] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.60728334, 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012027163, 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13677 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519343+0000", 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2599999999999998e-07 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519347+0000", 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9879999999999998e-06 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519351+0000", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7019999999999999e-06 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519353+0000", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7750000000000001e-06 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531247+0000", 2024-09-17T21:46:30.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011893124999999999 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531288+0000", 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1931000000000002e-05 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531370+0000", 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1216000000000003e-05 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:13678 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify cookie 140633671149968] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.607253462999999, 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012240881, 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13678 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519372+0000", 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519373+0000", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519375+0000", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519378+0000", 2024-09-17T21:46:30.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3610000000000001e-06 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.519381+0000", 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7010000000000001e-06 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531414+0000", 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012033333 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531482+0000", 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7958000000000002e-05 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.531613+0000", 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013111000000000001 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15175 3.15 3:ae5b5468:::rbd_data.11121f8e87bb.00000000000000f0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.234148542, 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011304611000000001, 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15175 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892477+0000", 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892478+0000", 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7599999999999997e-07 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892481+0000", 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4949999999999998e-06 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892486+0000", 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6849999999999998e-06 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.892489+0000", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2780000000000002e-06 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902702+0000", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01021262 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902774+0000", 2024-09-17T21:46:30.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1501999999999994e-05 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.902870+0000", 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6459000000000006e-05 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903650+0000", 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077978100000000001 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903743+0000", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3430000000000005e-05 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903764+0000", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0769999999999999e-05 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.903782+0000", 2024-09-17T21:46:30.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7915e-05 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15228 3.15 3:ab518994:::rbd_data.11121f8e87bb.0000000000000141:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.227654585, 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011106143000000001, 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15228 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898971+0000", 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898972+0000", 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.32e-07 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898974+0000", 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3999999999999999e-06 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898981+0000", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.387e-06 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.898985+0000", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7829999999999998e-06 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908707+0000", 2024-09-17T21:46:30.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097212989999999992 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908765+0000", 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8559e-05 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.908842+0000", 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6676999999999998e-05 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.909594+0000", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075257399999999999 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910038+0000", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000443831 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910059+0000", 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1189999999999999e-05 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.910077+0000", 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8011000000000001e-05 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3002 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.225670835000001, 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011091225999999999, 2024-09-17T21:46:30.199 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3002 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900955+0000", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900956+0000", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.269e-06 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900961+0000", 2024-09-17T21:46:30.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900972+0000", 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0575e-05 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.900979+0000", 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.911925+0000", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010946035 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.911977+0000", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1873000000000003e-05 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912046+0000", 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9536000000000006e-05 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15245 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.225553902, 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011088077999999999, 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15245 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901072+0000", 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901073+0000", 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6599999999999996e-07 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901076+0000", 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3220000000000001e-06 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901083+0000", 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.148e-06 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901087+0000", 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0509999999999998e-06 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912075+0000", 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010987393999999999 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912113+0000", 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8392000000000001e-05 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912160+0000", 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7104999999999997e-05 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3003 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.225540081, 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011227009, 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3003 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901086+0000", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901087+0000", 2024-09-17T21:46:30.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4900000000000005e-07 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901090+0000", 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7060000000000002e-06 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901098+0000", 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4610000000000002e-06 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901102+0000", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0679999999999998e-06 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912183+0000", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011081336000000001 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912219+0000", 2024-09-17T21:46:30.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5679000000000002e-05 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912313+0000", 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.391e-05 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:15246 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.2254009, 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011271723000000001, 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15246 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901225+0000", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901226+0000", 2024-09-17T21:46:30.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4000000000000001e-07 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901228+0000", 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.79e-06 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901234+0000", 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.755e-06 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901238+0000", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6969999999999999e-06 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912346+0000", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011108487 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912415+0000", 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8547999999999995e-05 2024-09-17T21:46:30.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912497+0000", 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1805999999999996e-05 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3004 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.225370149, 2024-09-17T21:46:30.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011465227999999999, 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3004 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901256+0000", 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901257+0000", 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9500000000000001e-07 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901260+0000", 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1099999999999999e-06 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901266+0000", 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9139999999999996e-06 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.901269+0000", 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7900000000000001e-06 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912527+0000", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011258068 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912586+0000", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9086000000000003e-05 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.912721+0000", 2024-09-17T21:46:30.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013446500000000001 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3017 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.22098853, 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011080370000000001, 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3017 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905637+0000", 2024-09-17T21:46:30.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905638+0000", 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6199999999999997e-07 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905641+0000", 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3390000000000001e-06 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905648+0000", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8399999999999997e-06 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905652+0000", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1219999999999997e-06 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916608+0000", 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010955998999999999 2024-09-17T21:46:30.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916652+0000", 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-05 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916718+0000", 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5518000000000005e-05 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:3018 3.d 3:b4d1219f:::rbd_header.11121f8e87bb:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.220836086, 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011167348000000001, 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3018 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905790+0000", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905791+0000", 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:46:30.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905795+0000", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4410000000000003e-06 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905805+0000", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3100000000000006e-06 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.905812+0000", 2024-09-17T21:46:30.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9310000000000002e-06 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916840+0000", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011028532000000001 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916882+0000", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1523999999999997e-05 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.916957+0000", 2024-09-17T21:46:30.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5378999999999998e-05 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.232 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:30.234 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:30.234 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:30.231+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:30.334 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:30.332+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:30.382 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:30.382 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:30.382 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:30.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:30.436 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:30.434+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:30.496 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:30.497 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:30.497 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:30.505 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:30.548 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:30.546+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.660042728000001, 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:30.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.658017676, 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:30.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:30.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.64403162, 2024-09-17T21:46:30.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:30.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.642070413000001, 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.629 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:30.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.640037056000001, 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.638059137999999, 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:30.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.638036344, 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:30.646 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.632127281000001, 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:30.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:30.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.630096222000001, 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.630069647999999, 2024-09-17T21:46:30.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:30.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.629459855, 2024-09-17T21:46:30.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:30.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:30.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:30.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.629442269, 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:30.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:30.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:30.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.629427312000001, 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.913745823999999, 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.913715203000001, 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:30.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:30.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:30.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:30.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:30.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.913690584999999, 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:30.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:30.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:30.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:30.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:30.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.913668638000001, 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.913647168000001, 2024-09-17T21:46:30.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:30.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:30.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8722303650000001, 2024-09-17T21:46:30.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:30.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8711859049999999, 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:30.717 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:30.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:30.719 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:30.716+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:30.724 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:46:30.728 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:46:30.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:46:30.808 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone7 from 0x2b1e600 (overlap 0x17c8c00) to 0x1567e00 2024-09-17T21:46:30.819 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:30.817+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:30.881 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:30.881 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:30.881 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:30.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:30.932 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:30.929+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:31.022 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:31.020+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:31.038 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:31.038 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:31.039 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:31.039 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:31.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:31.123 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:31.120+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:31.161 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:31.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.187041388000001, 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:31.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.187018895, 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:31.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:31.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186994635, 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:31.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:31.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186968752, 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:31.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:31.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:31.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186943750999999, 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:31.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186903952, 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:31.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186888207000001, 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:31.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186876672, 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.199 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.186866372000001, 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:31.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:31.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184895128999999, 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184881981, 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:31.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:31.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:31.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184871375, 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:31.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:31.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184861036999999, 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:31.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:31.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184851030000001, 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:31.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:31.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.229 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184841079, 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:31.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.184825771, 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:31.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:31.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.182849883999999, 2024-09-17T21:46:31.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:31.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:31.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.182838928000001, 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:31.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:31.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.182826438999999, 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:31.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.182815976000001, 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:31.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:31.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:31.258 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:31.260 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-17T21:46:31.260 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:31.257+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:31.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:31.360 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:31.358+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:31.386 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:31.387 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:31.387 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:31.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:31.461 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:31.459+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:31.504 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:31.505 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:31.506 INFO:teuthology.orchestra.run.smithi165.stdout:966 write 0x889d600 thru 0x889df1d (0x91e bytes) 2024-09-17T21:46:31.507 INFO:teuthology.orchestra.run.smithi165.stdout:967 write 0x289c200 thru 0x28a2361 (0x6162 bytes) 2024-09-17T21:46:31.508 INFO:teuthology.orchestra.run.smithi165.stdout:969 write 0x7c1c200 thru 0x7c27bbc (0xb9bd bytes) 2024-09-17T21:46:31.508 INFO:teuthology.orchestra.run.smithi165.stdout:970 read 0x609fe00 thru 0x60a3cf2 (0x3ef3 bytes) 2024-09-17T21:46:31.508 INFO:teuthology.orchestra.run.smithi165.stdout:971 read 0xcc9e00 thru 0xccc09c (0x229d bytes) 2024-09-17T21:46:31.508 INFO:teuthology.orchestra.run.smithi165.stdout:972 read 0x193c000 thru 0x193f2f4 (0x32f5 bytes) 2024-09-17T21:46:31.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:31.538 INFO:teuthology.orchestra.run.smithi165.stdout:973 write 0xe3cba00 thru 0xe3d882c (0xce2d bytes) 2024-09-17T21:46:31.538 INFO:teuthology.orchestra.run.smithi165.stdout:974 punch from 0x2268a00 to 0x2271000, (0x8600 bytes) 2024-09-17T21:46:31.546 INFO:teuthology.orchestra.run.smithi165.stdout:975 trunc from 0xe3d8a00 to 0x85cda00 2024-09-17T21:46:31.562 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:31.559+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:31.581 INFO:teuthology.orchestra.run.smithi165.stdout:977 write 0xe28e800 thru 0xe293bf0 (0x53f1 bytes) 2024-09-17T21:46:31.590 INFO:teuthology.orchestra.run.smithi165.stdout:979 trunc from 0xe293c00 to 0x6e76800 2024-09-17T21:46:31.607 INFO:teuthology.orchestra.run.smithi165.stdout:980 clone 9 order 21 su 262144 sc 5 2024-09-17T21:46:31.662 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:31.660+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:31.670 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.384131547999999, 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:31.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:31.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.362204744, 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:31.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:31.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.010816708, 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:31.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:31.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.50588759, 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.690 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.694 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.505877854000001, 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:31.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.505868242999998, 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:31.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.505857412000001, 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:31.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:31.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.505843820999999, 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:31.708 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:31.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.505833470999999, 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:31.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.500721093999999, 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:31.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.722 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.496394399, 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.723 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.727 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.496255189999999, 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.728 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:31.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:31.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.732 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.496238200000001, 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:31.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.496217202, 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:31.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:31.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.491371048000001, 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:31.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.491358639000001, 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:31.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.491348306999999, 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:31.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9841044110000006, 2024-09-17T21:46:31.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:31.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:31.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:31.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9518294360000006, 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:31.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:31.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:31.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.9515906679999997, 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:31.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:31.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:31.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:31.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:31.773 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:31.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:31.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:46:31.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:46:31.777 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:31.773+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:31.816 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone8 from 0x6e76800 (overlap 0x2b1e600) to 0x0 2024-09-17T21:46:31.876 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:31.873+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:31.977 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:31.975+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:32.000 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:32.000 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4918 3.b 3:d33a2876:::rbd_header.1112a8bc19be:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.998452+0000", 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00019692000000000001, 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022408700000000001, 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:32.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4918 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998452+0000", 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998452+0000", 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:32.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998453+0000", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998456+0000", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6090000000000001e-06 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998466+0000", 2024-09-17T21:46:32.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5429999999999999e-06 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998473+0000", 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0949999999999998e-06 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998530+0000", 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6922999999999997e-05 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:32.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.998606+0000", 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6266999999999993e-05 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:46:32.005 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:32.010 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:32.014 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:32.032 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:32.032 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:32.051 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:32.051 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:32.069 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:32.069 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:32.077 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:32.075+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:32.086 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:32.104 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:32.104 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:32.128 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.128 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:32.178 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:32.175+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:32.278 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:32.276+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:32.286 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.287 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:32.314 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-17T21:46:32.379 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:32.377+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:32.423 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.423 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.435 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:32.480 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:32.478+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:32.581 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:32.579+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:32.585 INFO:teuthology.orchestra.run.smithi165.stdout:981 write 0x5e5fe00 thru 0x5e6a777 (0xa978 bytes) 2024-09-17T21:46:32.586 INFO:teuthology.orchestra.run.smithi165.stdout:982 read 0x5d2f800 thru 0x5d36e11 (0x7612 bytes) 2024-09-17T21:46:32.586 INFO:teuthology.orchestra.run.smithi165.stdout:983 read 0x4695c00 thru 0x469d293 (0x7694 bytes) 2024-09-17T21:46:32.587 INFO:teuthology.orchestra.run.smithi165.stdout:985 read 0x5b0d400 thru 0x5b16865 (0x9466 bytes) 2024-09-17T21:46:32.587 INFO:teuthology.orchestra.run.smithi165.stdout:986 read 0x22bf200 thru 0x22c4716 (0x5517 bytes) 2024-09-17T21:46:32.588 INFO:teuthology.orchestra.run.smithi165.stdout:987 write 0x4044c00 thru 0x4051360 (0xc761 bytes) 2024-09-17T21:46:32.590 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.590 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.595 INFO:teuthology.orchestra.run.smithi165.stdout:991 trunc from 0x6e76800 to 0xc5eba00 2024-09-17T21:46:32.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:32.616 INFO:teuthology.orchestra.run.smithi165.stdout:994 trunc from 0xc5eba00 to 0x9ddfe00 2024-09-17T21:46:32.624 INFO:teuthology.orchestra.run.smithi165.stdout:995 read 0x9526800 thru 0x95302e4 (0x9ae5 bytes) 2024-09-17T21:46:32.630 INFO:teuthology.orchestra.run.smithi165.stdout:997 trunc from 0x9ddfe00 to 0x5da0400 2024-09-17T21:46:32.645 INFO:teuthology.orchestra.run.smithi165.stdout:999 write 0x38ee400 thru 0x38f14c8 (0x30c9 bytes) 2024-09-17T21:46:32.647 INFO:teuthology.orchestra.run.smithi165.stdout:1000 punch from 0x5d0b400 to 0x5d16e00, (0xba00 bytes) 2024-09-17T21:46:32.647 INFO:teuthology.orchestra.run.smithi165.stdout:1009 read 0x1428800 thru 0x142acda (0x24db bytes) 2024-09-17T21:46:32.648 INFO:teuthology.orchestra.run.smithi165.stdout:1010 read 0x416de00 thru 0x417d936 (0xfb37 bytes) 2024-09-17T21:46:32.651 INFO:teuthology.orchestra.run.smithi165.stdout:1012 trunc from 0x5da0400 to 0x8005000 2024-09-17T21:46:32.661 INFO:teuthology.orchestra.run.smithi165.stdout:1014 write 0x1d95a00 thru 0x1d98af8 (0x30f9 bytes) 2024-09-17T21:46:32.661 INFO:teuthology.orchestra.run.smithi165.stdout:1015 read 0x724cc00 thru 0x725704b (0xa44c bytes) 2024-09-17T21:46:32.668 INFO:teuthology.orchestra.run.smithi165.stdout:1017 trunc from 0x8005000 to 0x220a200 2024-09-17T21:46:32.683 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:32.680+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:32.714 INFO:teuthology.orchestra.run.smithi165.stdout:1018 read 0x1d51200 thru 0x1d557ee (0x45ef bytes) 2024-09-17T21:46:32.729 INFO:teuthology.orchestra.run.smithi165.stdout:1020 write 0x6d61e00 thru 0x6d6a24c (0x844d bytes) 2024-09-17T21:46:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:46:32.734 INFO:teuthology.orchestra.run.smithi165.stdout:1021 read 0x4431400 thru 0x4438a97 (0x7698 bytes) 2024-09-17T21:46:32.734 INFO:teuthology.orchestra.run.smithi165.stdout:1022 clone 10 order 25 su 524288 sc 4 2024-09-17T21:46:32.743 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:46:32.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:46:32.764 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.764 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:32.814 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:32.812+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:32.880 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.880 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:32.884 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:32.881+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:32.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:32.985 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:32.982+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:32.995 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:32.995 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:33.086 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:33.084+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:33.151 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.151 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:33.203 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:33.201+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:33.266 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.267 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.276 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:33.316 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:33.314+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:33.389 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:33.387+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:33.422 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.422 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.431 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:33.490 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:33.487+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:33.537 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.537 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.546 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:33.630 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:33.628+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:33.691 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:33.689+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:33.693 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.693 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:33.737 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:46:33.740 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:46:33.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:46:33.792 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:33.789+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:33.842 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone9 intact 2024-09-17T21:46:33.851 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:33.851 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:33.860 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:33.892 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:33.890+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:33.993 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:33.991+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:34.013 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:34.014 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:34.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:34.094 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:34.092+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:34.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:34.133 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:34.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:34.213 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-17T21:46:34.228 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:34.226+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:34.243 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:34.244 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:34.260 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:34.260 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:34.270 INFO:teuthology.orchestra.run.smithi165.stdout:1023 punch from 0x6c03e00 to 0x6c06800, (0x2a00 bytes) 2024-09-17T21:46:34.284 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:34.296 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:34.293+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:34.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:34.301 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:34.301 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:34.301 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:34.302 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:34.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:34.320 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:34.327 INFO:teuthology.orchestra.run.smithi165.stdout:1024 punch from 0x54e5600 to 0x54f4000, (0xea00 bytes) 2024-09-17T21:46:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:34.344 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:34.378 INFO:teuthology.orchestra.run.smithi165.stdout:1025 punch from 0x277c000 to 0x277d200, (0x1200 bytes) 2024-09-17T21:46:34.397 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:34.395+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:34.419 INFO:teuthology.orchestra.run.smithi165.stdout:1028 clone 11 order 24 su 4194304 sc 11 2024-09-17T21:46:34.441 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.154721435999999, 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.132794632, 2024-09-17T21:46:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.781406596, 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276477478, 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276467742000001, 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276458130999998, 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276447300000001, 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276433708999999, 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.276423358999999, 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.271310981999999, 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.266984287, 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.266845077999999, 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.266828088, 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.26680709, 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.261960936000001, 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.261948527000001, 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.261938194999999, 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.754694299000001, 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:34.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:34.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.722419324000001, 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:34.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:34.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.722180556, 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:34.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:34.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:34.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.545 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:34.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:34.546 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:34.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:34.548 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:34.544+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:34.648 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:34.646+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:34.695 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:34.695 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:34.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:34.739 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:46:34.755 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:46:34.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:46:34.755 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:34.752+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:34.848 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone10 from 0x6d6a400 (overlap 0x6d6a400) to 0x6279200 2024-09-17T21:46:34.849 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:34.847+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:34.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:34.855 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:34.855 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:34.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:34.950 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:34.947+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112899006999999, 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:35.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.025 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112888706, 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112872631999998, 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112861419000001, 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112843981000001, 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112833111, 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.112638233999999, 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.110933564, 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.110922926000001, 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.110912833, 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.110902714000002, 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.108772483999999, 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.108762753000001, 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.107979394000001, 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.107963823999999, 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.107870922, 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.107376302999999, 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.107232258, 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:35.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.106546820999998, 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:35.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:35.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.106184430999999, 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.113 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:35.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:35.116 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:46:35.116 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:46:35.117 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:35.112+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:35.120 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-17T21:46:34.736832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506801344,"num_objects":447,"num_object_clones":79,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":107,"num_read":68194,"num_read_kb":305458,"num_write":5398,"num_write_kb":162109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":24842019,"num_keys_recovered":158,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4273,"ondisk_log_size":4273,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":450244,"kb_used_data":252676,"kb_used_omap":7,"kb_used_meta":197432,"kb_avail":471408956,"statfs":{"total":483183820800,"available":482722770944,"internally_reserved":0,"allocated":258740224,"data_stored":445859634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7973,"internal_metadata":202170587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11614759,"num_objects":75,"num_object_clones":23,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":18,"num_read":11086,"num_read_kb":54500,"num_write":692,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1490503,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013833"},"pg_stats":[{"pgid":"3.1f","version":"47'90","reported_seq":1260,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108129+0000","last_change":"2024-09-17T21:46:25.785411+0000","last_active":"2024-09-17T21:46:31.108129+0000","last_peered":"2024-09-17T21:46:31.108129+0000","last_clean":"2024-09-17T21:46:31.108129+0000","last_became_active":"2024-09-17T21:46:25.711785+0000","last_became_peered":"2024-09-17T21:46:25.711785+0000","last_unstale":"2024-09-17T21:46:31.108129+0000","last_undegraded":"2024-09-17T21:46:31.108129+0000","last_fullsized":"2024-09-17T21:46:31.108129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:11.929397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":18236981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":2146,"num_read_kb":10723,"num_write":82,"num_write_kb":2965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":6680735,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"47'149","reported_seq":4550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724927+0000","last_change":"2024-09-17T21:46:03.271053+0000","last_active":"2024-09-17T21:46:30.724927+0000","last_peered":"2024-09-17T21:46:30.724927+0000","last_clean":"2024-09-17T21:46:30.724927+0000","last_became_active":"2024-09-17T21:46:01.267064+0000","last_became_peered":"2024-09-17T21:46:01.267064+0000","last_unstale":"2024-09-17T21:46:30.724927+0000","last_undegraded":"2024-09-17T21:46:30.724927+0000","last_fullsized":"2024-09-17T21:46:30.724927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:59:13.016540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":49006592,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":4369,"num_read_kb":27584,"num_write":135,"num_write_kb":16082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"46'45","reported_seq":985,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.863488+0000","last_change":"2024-09-17T21:46:03.281927+0000","last_active":"2024-09-17T21:46:30.863488+0000","last_peered":"2024-09-17T21:46:30.863488+0000","last_clean":"2024-09-17T21:46:30.863488+0000","last_became_active":"2024-09-17T21:46:01.272354+0000","last_became_peered":"2024-09-17T21:46:01.272354+0000","last_unstale":"2024-09-17T21:46:30.863488+0000","last_undegraded":"2024-09-17T21:46:30.863488+0000","last_fullsized":"2024-09-17T21:46:30.863488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:42.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":11212288,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":883,"num_read_kb":4669,"num_write":64,"num_write_kb":3555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'70","reported_seq":2483,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108131+0000","last_change":"2024-09-17T21:46:03.282883+0000","last_active":"2024-09-17T21:46:31.108131+0000","last_peered":"2024-09-17T21:46:31.108131+0000","last_clean":"2024-09-17T21:46:31.108131+0000","last_became_active":"2024-09-17T21:46:01.270813+0000","last_became_peered":"2024-09-17T21:46:01.270813+0000","last_unstale":"2024-09-17T21:46:31.108131+0000","last_undegraded":"2024-09-17T21:46:31.108131+0000","last_fullsized":"2024-09-17T21:46:31.108131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:08:53.860488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":26124288,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":2376,"num_read_kb":20211,"num_write":130,"num_write_kb":21387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'186","reported_seq":3140,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.550064+0000","last_change":"2024-09-17T21:46:28.714405+0000","last_active":"2024-09-17T21:46:29.550064+0000","last_peered":"2024-09-17T21:46:29.550064+0000","last_clean":"2024-09-17T21:46:29.550064+0000","last_became_active":"2024-09-17T21:46:28.713834+0000","last_became_peered":"2024-09-17T21:46:28.713834+0000","last_unstale":"2024-09-17T21:46:29.550064+0000","last_undegraded":"2024-09-17T21:46:29.550064+0000","last_fullsized":"2024-09-17T21:46:29.550064+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:00.307070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":39733760,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":4511,"num_read_kb":10168,"num_write":124,"num_write_kb":1776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3707392,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"46'36","reported_seq":1492,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.762958+0000","last_change":"2024-09-17T21:46:03.277187+0000","last_active":"2024-09-17T21:46:29.762958+0000","last_peered":"2024-09-17T21:46:29.762958+0000","last_clean":"2024-09-17T21:46:29.762958+0000","last_became_active":"2024-09-17T21:46:01.268588+0000","last_became_peered":"2024-09-17T21:46:01.268588+0000","last_unstale":"2024-09-17T21:46:29.762958+0000","last_undegraded":"2024-09-17T21:46:29.762958+0000","last_fullsized":"2024-09-17T21:46:29.762958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:33.186514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":14389354,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1407,"num_read_kb":9401,"num_write":39,"num_write_kb":2829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"48'120","reported_seq":1986,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737650+0000","last_change":"2024-09-17T21:46:03.278649+0000","last_active":"2024-09-17T21:46:33.737650+0000","last_peered":"2024-09-17T21:46:33.737650+0000","last_clean":"2024-09-17T21:46:33.737650+0000","last_became_active":"2024-09-17T21:46:01.267429+0000","last_became_peered":"2024-09-17T21:46:01.267429+0000","last_unstale":"2024-09-17T21:46:33.737650+0000","last_undegraded":"2024-09-17T21:46:33.737650+0000","last_fullsized":"2024-09-17T21:46:33.737650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:01.981707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":4857344,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1810,"num_read_kb":7110,"num_write":118,"num_write_kb":1691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"47'123","reported_seq":2349,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.848177+0000","last_change":"2024-09-17T21:46:28.714495+0000","last_active":"2024-09-17T21:46:30.848177+0000","last_peered":"2024-09-17T21:46:30.848177+0000","last_clean":"2024-09-17T21:46:30.848177+0000","last_became_active":"2024-09-17T21:46:28.713898+0000","last_became_peered":"2024-09-17T21:46:28.713898+0000","last_unstale":"2024-09-17T21:46:30.848177+0000","last_undegraded":"2024-09-17T21:46:30.848177+0000","last_fullsized":"2024-09-17T21:46:30.848177+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:36.307253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":34762240,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2318,"num_read_kb":4174,"num_write":106,"num_write_kb":1858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1621504,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'78","reported_seq":860,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108495+0000","last_change":"2024-09-17T21:46:25.754859+0000","last_active":"2024-09-17T21:46:31.108495+0000","last_peered":"2024-09-17T21:46:31.108495+0000","last_clean":"2024-09-17T21:46:31.108495+0000","last_became_active":"2024-09-17T21:46:25.709818+0000","last_became_peered":"2024-09-17T21:46:25.709818+0000","last_unstale":"2024-09-17T21:46:31.108495+0000","last_undegraded":"2024-09-17T21:46:31.108495+0000","last_fullsized":"2024-09-17T21:46:31.108495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:52.086875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":5029888,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":913,"num_read_kb":4444,"num_write":95,"num_write_kb":2566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2538496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"46'39","reported_seq":500,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106630+0000","last_change":"2024-09-17T21:46:03.282056+0000","last_active":"2024-09-17T21:46:31.106630+0000","last_peered":"2024-09-17T21:46:31.106630+0000","last_clean":"2024-09-17T21:46:31.106630+0000","last_became_active":"2024-09-17T21:46:01.267932+0000","last_became_peered":"2024-09-17T21:46:01.267932+0000","last_unstale":"2024-09-17T21:46:31.106630+0000","last_undegraded":"2024-09-17T21:46:31.106630+0000","last_fullsized":"2024-09-17T21:46:31.106630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:33.448348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":4932608,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":393,"num_read_kb":1492,"num_write":60,"num_write_kb":2763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"47'150","reported_seq":2743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.061347+0000","last_change":"2024-09-17T21:46:25.826556+0000","last_active":"2024-09-17T21:46:31.061347+0000","last_peered":"2024-09-17T21:46:31.061347+0000","last_clean":"2024-09-17T21:46:31.061347+0000","last_became_active":"2024-09-17T21:46:25.716440+0000","last_became_peered":"2024-09-17T21:46:25.716440+0000","last_unstale":"2024-09-17T21:46:31.061347+0000","last_undegraded":"2024-09-17T21:46:31.061347+0000","last_fullsized":"2024-09-17T21:46:31.061347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:49.482394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":2616832,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2615,"num_read_kb":5156,"num_write":122,"num_write_kb":1079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1094656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"47'68","reported_seq":1913,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724609+0000","last_change":"2024-09-17T21:46:03.271246+0000","last_active":"2024-09-17T21:46:30.724609+0000","last_peered":"2024-09-17T21:46:30.724609+0000","last_clean":"2024-09-17T21:46:30.724609+0000","last_became_active":"2024-09-17T21:46:01.266638+0000","last_became_peered":"2024-09-17T21:46:01.266638+0000","last_unstale":"2024-09-17T21:46:30.724609+0000","last_undegraded":"2024-09-17T21:46:30.724609+0000","last_fullsized":"2024-09-17T21:46:30.724609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:50:18.206664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":17264656,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":1800,"num_read_kb":9798,"num_write":73,"num_write_kb":2206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724612+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:46:30.724612+0000","last_peered":"2024-09-17T21:46:30.724612+0000","last_clean":"2024-09-17T21:46:30.724612+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:46:30.724612+0000","last_undegraded":"2024-09-17T21:46:30.724612+0000","last_fullsized":"2024-09-17T21:46:30.724612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'87","reported_seq":1855,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.916766+0000","last_change":"2024-09-17T21:46:25.802846+0000","last_active":"2024-09-17T21:46:33.916766+0000","last_peered":"2024-09-17T21:46:33.916766+0000","last_clean":"2024-09-17T21:46:33.916766+0000","last_became_active":"2024-09-17T21:46:25.713238+0000","last_became_peered":"2024-09-17T21:46:25.713238+0000","last_unstale":"2024-09-17T21:46:33.916766+0000","last_undegraded":"2024-09-17T21:46:33.916766+0000","last_fullsized":"2024-09-17T21:46:33.916766+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:45.219111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":11810855,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1523,"num_read_kb":11744,"num_write":120,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3456117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737904+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:46:33.737904+0000","last_peered":"2024-09-17T21:46:33.737904+0000","last_clean":"2024-09-17T21:46:33.737904+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:46:33.737904+0000","last_undegraded":"2024-09-17T21:46:33.737904+0000","last_fullsized":"2024-09-17T21:46:33.737904+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'80","reported_seq":1430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.861951+0000","last_change":"2024-09-17T21:46:03.270712+0000","last_active":"2024-09-17T21:46:30.861951+0000","last_peered":"2024-09-17T21:46:30.861951+0000","last_clean":"2024-09-17T21:46:30.861951+0000","last_became_active":"2024-09-17T21:46:01.265693+0000","last_became_peered":"2024-09-17T21:46:01.265693+0000","last_unstale":"2024-09-17T21:46:30.861951+0000","last_undegraded":"2024-09-17T21:46:30.861951+0000","last_fullsized":"2024-09-17T21:46:30.861951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:10.796820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":3275776,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":1291,"num_read_kb":5510,"num_write":75,"num_write_kb":694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724649+0000","last_change":"2024-09-17T21:46:25.714843+0000","last_active":"2024-09-17T21:46:30.724649+0000","last_peered":"2024-09-17T21:46:30.724649+0000","last_clean":"2024-09-17T21:46:30.724649+0000","last_became_active":"2024-09-17T21:46:25.714521+0000","last_became_peered":"2024-09-17T21:46:25.714521+0000","last_unstale":"2024-09-17T21:46:30.724649+0000","last_undegraded":"2024-09-17T21:46:30.724649+0000","last_fullsized":"2024-09-17T21:46:30.724649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:05.250233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"48'78","reported_seq":2513,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737911+0000","last_change":"2024-09-17T21:46:03.279109+0000","last_active":"2024-09-17T21:46:33.737911+0000","last_peered":"2024-09-17T21:46:33.737911+0000","last_clean":"2024-09-17T21:46:33.737911+0000","last_became_active":"2024-09-17T21:46:01.268063+0000","last_became_peered":"2024-09-17T21:46:01.268063+0000","last_unstale":"2024-09-17T21:46:33.737911+0000","last_undegraded":"2024-09-17T21:46:33.737911+0000","last_fullsized":"2024-09-17T21:46:33.737911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:50.614943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":6665216,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2369,"num_read_kb":11942,"num_write":72,"num_write_kb":1416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":96,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.738193+0000","last_change":"2024-09-17T21:46:25.712667+0000","last_active":"2024-09-17T21:46:33.738193+0000","last_peered":"2024-09-17T21:46:33.738193+0000","last_clean":"2024-09-17T21:46:33.738193+0000","last_became_active":"2024-09-17T21:46:25.712313+0000","last_became_peered":"2024-09-17T21:46:25.712313+0000","last_unstale":"2024-09-17T21:46:33.738193+0000","last_undegraded":"2024-09-17T21:46:33.738193+0000","last_fullsized":"2024-09-17T21:46:33.738193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:06.825437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"47'69","reported_seq":952,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.095085+0000","last_change":"2024-09-17T21:46:03.282656+0000","last_active":"2024-09-17T21:46:31.095085+0000","last_peered":"2024-09-17T21:46:31.095085+0000","last_clean":"2024-09-17T21:46:31.095085+0000","last_became_active":"2024-09-17T21:46:01.271756+0000","last_became_peered":"2024-09-17T21:46:01.271756+0000","last_unstale":"2024-09-17T21:46:31.095085+0000","last_undegraded":"2024-09-17T21:46:31.095085+0000","last_fullsized":"2024-09-17T21:46:31.095085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:36.282504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":1556512,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":832,"num_read_kb":4097,"num_write":70,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"46'86","reported_seq":1082,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.636993+0000","last_change":"2024-09-17T21:46:25.744899+0000","last_active":"2024-09-17T21:46:29.636993+0000","last_peered":"2024-09-17T21:46:29.636993+0000","last_clean":"2024-09-17T21:46:29.636993+0000","last_became_active":"2024-09-17T21:46:25.713298+0000","last_became_peered":"2024-09-17T21:46:25.713298+0000","last_unstale":"2024-09-17T21:46:29.636993+0000","last_undegraded":"2024-09-17T21:46:29.636993+0000","last_fullsized":"2024-09-17T21:46:29.636993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:56.686749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":2912256,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":890,"num_read_kb":1261,"num_write":74,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":60416,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.729850+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:46:30.729850+0000","last_peered":"2024-09-17T21:46:30.729850+0000","last_clean":"2024-09-17T21:46:30.729850+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:46:30.729850+0000","last_undegraded":"2024-09-17T21:46:30.729850+0000","last_fullsized":"2024-09-17T21:46:30.729850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"48'105","reported_seq":2122,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.917661+0000","last_change":"2024-09-17T21:46:03.278873+0000","last_active":"2024-09-17T21:46:33.917661+0000","last_peered":"2024-09-17T21:46:33.917661+0000","last_clean":"2024-09-17T21:46:33.917661+0000","last_became_active":"2024-09-17T21:46:01.266414+0000","last_became_peered":"2024-09-17T21:46:01.266414+0000","last_unstale":"2024-09-17T21:46:33.917661+0000","last_undegraded":"2024-09-17T21:46:33.917661+0000","last_fullsized":"2024-09-17T21:46:33.917661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:02.782507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":15669371,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":2801,"num_read_kb":6611,"num_write":102,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.725191+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:46:30.725191+0000","last_peered":"2024-09-17T21:46:30.725191+0000","last_clean":"2024-09-17T21:46:30.725191+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:46:30.725191+0000","last_undegraded":"2024-09-17T21:46:30.725191+0000","last_fullsized":"2024-09-17T21:46:30.725191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'1021","reported_seq":1818,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.800543+0000","last_change":"2024-09-17T21:46:28.714716+0000","last_active":"2024-09-17T21:46:29.800543+0000","last_peered":"2024-09-17T21:46:29.800543+0000","last_clean":"2024-09-17T21:46:29.800543+0000","last_became_active":"2024-09-17T21:46:28.713879+0000","last_became_peered":"2024-09-17T21:46:28.713879+0000","last_unstale":"2024-09-17T21:46:29.800543+0000","last_undegraded":"2024-09-17T21:46:29.800543+0000","last_fullsized":"2024-09-17T21:46:29.800543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:12.313725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":3885823,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":6173,"num_read_kb":5971,"num_write":1795,"num_write_kb":2161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":98560,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724256+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:46:30.724256+0000","last_peered":"2024-09-17T21:46:30.724256+0000","last_clean":"2024-09-17T21:46:30.724256+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:46:30.724256+0000","last_undegraded":"2024-09-17T21:46:30.724256+0000","last_fullsized":"2024-09-17T21:46:30.724256+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"48'78","reported_seq":2372,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737975+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:46:33.737975+0000","last_peered":"2024-09-17T21:46:33.737975+0000","last_clean":"2024-09-17T21:46:33.737975+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:46:33.737975+0000","last_undegraded":"2024-09-17T21:46:33.737975+0000","last_fullsized":"2024-09-17T21:46:33.737975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:11.430369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":31046672,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2241,"num_read_kb":20787,"num_write":73,"num_write_kb":16521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.729846+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:46:30.729846+0000","last_peered":"2024-09-17T21:46:30.729846+0000","last_clean":"2024-09-17T21:46:30.729846+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:46:30.729846+0000","last_undegraded":"2024-09-17T21:46:30.729846+0000","last_fullsized":"2024-09-17T21:46:30.729846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724259+0000","last_change":"2024-09-17T21:46:28.714299+0000","last_active":"2024-09-17T21:46:30.724259+0000","last_peered":"2024-09-17T21:46:30.724259+0000","last_clean":"2024-09-17T21:46:30.724259+0000","last_became_active":"2024-09-17T21:46:28.713856+0000","last_became_peered":"2024-09-17T21:46:28.713856+0000","last_unstale":"2024-09-17T21:46:30.724259+0000","last_undegraded":"2024-09-17T21:46:30.724259+0000","last_fullsized":"2024-09-17T21:46:30.724259+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:14.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"47'85","reported_seq":2229,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.861982+0000","last_change":"2024-09-17T21:46:03.270987+0000","last_active":"2024-09-17T21:46:30.861982+0000","last_peered":"2024-09-17T21:46:30.861982+0000","last_clean":"2024-09-17T21:46:30.861982+0000","last_became_active":"2024-09-17T21:46:01.270323+0000","last_became_peered":"2024-09-17T21:46:01.270323+0000","last_unstale":"2024-09-17T21:46:30.861982+0000","last_undegraded":"2024-09-17T21:46:30.861982+0000","last_fullsized":"2024-09-17T21:46:30.861982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:46.882120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":15392784,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2091,"num_read_kb":9919,"num_write":80,"num_write_kb":694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"46'117","reported_seq":2177,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.774735+0000","last_change":"2024-09-17T21:46:03.276671+0000","last_active":"2024-09-17T21:46:29.774735+0000","last_peered":"2024-09-17T21:46:29.774735+0000","last_clean":"2024-09-17T21:46:29.774735+0000","last_became_active":"2024-09-17T21:46:01.263881+0000","last_became_peered":"2024-09-17T21:46:01.263881+0000","last_unstale":"2024-09-17T21:46:29.774735+0000","last_undegraded":"2024-09-17T21:46:29.774735+0000","last_fullsized":"2024-09-17T21:46:29.774735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.049396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":34484260,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2242,"num_read_kb":19651,"num_write":166,"num_write_kb":19163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'73","reported_seq":1457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106992+0000","last_change":"2024-09-17T21:46:03.271833+0000","last_active":"2024-09-17T21:46:31.106992+0000","last_peered":"2024-09-17T21:46:31.106992+0000","last_clean":"2024-09-17T21:46:31.106992+0000","last_became_active":"2024-09-17T21:46:01.268770+0000","last_became_peered":"2024-09-17T21:46:01.268770+0000","last_unstale":"2024-09-17T21:46:31.106992+0000","last_undegraded":"2024-09-17T21:46:31.106992+0000","last_fullsized":"2024-09-17T21:46:31.106992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:04.211419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":3338240,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1330,"num_read_kb":4321,"num_write":70,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'104","reported_seq":2673,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:34.087171+0000","last_change":"2024-09-17T21:46:25.763286+0000","last_active":"2024-09-17T21:46:34.087171+0000","last_peered":"2024-09-17T21:46:34.087171+0000","last_clean":"2024-09-17T21:46:34.087171+0000","last_became_active":"2024-09-17T21:46:25.712470+0000","last_became_peered":"2024-09-17T21:46:25.712470+0000","last_unstale":"2024-09-17T21:46:34.087171+0000","last_undegraded":"2024-09-17T21:46:34.087171+0000","last_fullsized":"2024-09-17T21:46:34.087171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:21.259438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":11227152,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2563,"num_read_kb":13195,"num_write":118,"num_write_kb":3358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2535440,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"47'122","reported_seq":1559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.730003+0000","last_change":"2024-09-17T21:46:28.714996+0000","last_active":"2024-09-17T21:46:30.730003+0000","last_peered":"2024-09-17T21:46:30.730003+0000","last_clean":"2024-09-17T21:46:30.730003+0000","last_became_active":"2024-09-17T21:46:28.714320+0000","last_became_peered":"2024-09-17T21:46:28.714320+0000","last_unstale":"2024-09-17T21:46:30.730003+0000","last_undegraded":"2024-09-17T21:46:30.730003+0000","last_fullsized":"2024-09-17T21:46:30.730003+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:21.360651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":13533223,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1309,"num_read_kb":6849,"num_write":141,"num_write_kb":450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":386165,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'258","reported_seq":4862,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.107473+0000","last_change":"2024-09-17T21:46:03.272242+0000","last_active":"2024-09-17T21:46:31.107473+0000","last_peered":"2024-09-17T21:46:31.107473+0000","last_clean":"2024-09-17T21:46:31.107473+0000","last_became_active":"2024-09-17T21:46:01.268272+0000","last_became_peered":"2024-09-17T21:46:01.268272+0000","last_unstale":"2024-09-17T21:46:31.107473+0000","last_undegraded":"2024-09-17T21:46:31.107473+0000","last_fullsized":"2024-09-17T21:46:31.107473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:40.281908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":14970918,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":4763,"num_read_kb":7739,"num_write":232,"num_write_kb":3959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'55","reported_seq":373,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.829612+0000","last_change":"2024-09-17T21:46:03.283005+0000","last_active":"2024-09-17T21:46:30.829612+0000","last_peered":"2024-09-17T21:46:30.829612+0000","last_clean":"2024-09-17T21:46:30.829612+0000","last_became_active":"2024-09-17T21:46:01.272460+0000","last_became_peered":"2024-09-17T21:46:01.272460+0000","last_unstale":"2024-09-17T21:46:30.829612+0000","last_undegraded":"2024-09-17T21:46:30.829612+0000","last_fullsized":"2024-09-17T21:46:30.829612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:01.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":4431872,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":252,"num_read_kb":932,"num_write":55,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"47'282","reported_seq":3625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106725+0000","last_change":"2024-09-17T21:46:03.282498+0000","last_active":"2024-09-17T21:46:31.106725+0000","last_peered":"2024-09-17T21:46:31.106725+0000","last_clean":"2024-09-17T21:46:31.106725+0000","last_became_active":"2024-09-17T21:46:01.268915+0000","last_became_peered":"2024-09-17T21:46:01.268915+0000","last_unstale":"2024-09-17T21:46:31.106725+0000","last_undegraded":"2024-09-17T21:46:31.106725+0000","last_fullsized":"2024-09-17T21:46:31.106725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:22:24.434413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":28082762,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":4364,"num_read_kb":27601,"num_write":501,"num_write_kb":23495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"47'61","reported_seq":2935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108260+0000","last_change":"2024-09-17T21:46:25.778540+0000","last_active":"2024-09-17T21:46:31.108260+0000","last_peered":"2024-09-17T21:46:31.108260+0000","last_clean":"2024-09-17T21:46:31.108260+0000","last_became_active":"2024-09-17T21:46:25.711546+0000","last_became_peered":"2024-09-17T21:46:25.711546+0000","last_unstale":"2024-09-17T21:46:31.108260+0000","last_undegraded":"2024-09-17T21:46:31.108260+0000","last_fullsized":"2024-09-17T21:46:31.108260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:54.073920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":20673145,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2616,"num_read_kb":10370,"num_write":69,"num_write_kb":1195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":883242,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"47'90","reported_seq":1369,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.828655+0000","last_change":"2024-09-17T21:46:03.271231+0000","last_active":"2024-09-17T21:46:30.828655+0000","last_peered":"2024-09-17T21:46:30.828655+0000","last_clean":"2024-09-17T21:46:30.828655+0000","last_became_active":"2024-09-17T21:46:01.266566+0000","last_became_peered":"2024-09-17T21:46:01.266566+0000","last_unstale":"2024-09-17T21:46:30.828655+0000","last_undegraded":"2024-09-17T21:46:30.828655+0000","last_fullsized":"2024-09-17T21:46:30.828655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:56.880646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":19168256,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1226,"num_read_kb":15767,"num_write":87,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'51","reported_seq":313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724349+0000","last_change":"2024-09-17T21:46:25.734932+0000","last_active":"2024-09-17T21:46:30.724349+0000","last_peered":"2024-09-17T21:46:30.724349+0000","last_clean":"2024-09-17T21:46:30.724349+0000","last_became_active":"2024-09-17T21:46:25.710701+0000","last_became_peered":"2024-09-17T21:46:25.710701+0000","last_unstale":"2024-09-17T21:46:30.724349+0000","last_undegraded":"2024-09-17T21:46:30.724349+0000","last_fullsized":"2024-09-17T21:46:30.724349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:41.751095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":10022912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":160,"num_read_kb":817,"num_write":47,"num_write_kb":226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"46'97","reported_seq":1475,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108787+0000","last_change":"2024-09-17T21:46:03.272109+0000","last_active":"2024-09-17T21:46:31.108787+0000","last_peered":"2024-09-17T21:46:31.108787+0000","last_clean":"2024-09-17T21:46:31.108787+0000","last_became_active":"2024-09-17T21:46:01.267697+0000","last_became_peered":"2024-09-17T21:46:01.267697+0000","last_unstale":"2024-09-17T21:46:31.108787+0000","last_undegraded":"2024-09-17T21:46:31.108787+0000","last_fullsized":"2024-09-17T21:46:31.108787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:38.773233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":25896121,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1501,"num_read_kb":5344,"num_write":142,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":506210957,"num_objects":443,"num_object_clones":79,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":107,"num_read":68078,"num_read_kb":305358,"num_write":5237,"num_write_kb":159691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":23070915,"num_keys_recovered":158,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":257904640,"data_stored":446764128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4153,"ondisk_log_size":4153,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607565,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95864,"kb_used_data":60520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275976,"statfs":{"total":96636764160,"available":96538599424,"internally_reserved":0,"allocated":61972480,"data_stored":114512194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607564,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99248,"kb_used_data":59744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94272592,"statfs":{"total":96636764160,"available":96535134208,"internally_reserved":0,"allocated":61177856,"data_stored":107276640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607564,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83416,"kb_used_data":39720,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94288424,"statfs":{"total":96636764160,"available":96551346176,"internally_reserved":0,"allocated":40673280,"data_stored":68831838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607564,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99324,"kb_used_data":59816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94272516,"statfs":{"total":96636764160,"available":96535056384,"internally_reserved":0,"allocated":61251584,"data_stored":106780538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":32876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":33665024,"data_stored":48458424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2244608,"data_stored":2244390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32505856,"data_stored":47648181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40148992,"data_stored":68627481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":60080128,"data_stored":106560965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61460480,"data_stored":107304404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61464576,"data_stored":114378707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:46:35.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:46:35.215 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:35.212+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:35.273 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:35.273 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:35.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:35.326 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:35.324+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:35.356 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-17T21:46:35.401 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:35.401 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:35.402 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:35.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:35.416 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:35.414+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:46:35.497 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:46:35.512 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-17T21:46:34.736832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":506801344,"num_objects":447,"num_object_clones":79,"num_object_copies":894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":447,"num_whiteouts":107,"num_read":68194,"num_read_kb":305458,"num_write":5398,"num_write_kb":162109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":538,"num_bytes_recovered":24842019,"num_keys_recovered":158,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4273,"ondisk_log_size":4273,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":81,"num_osds":5,"num_per_pool_osds":5,"num_per_pool_omap_osds":5,"kb":471859200,"kb_used":450244,"kb_used_data":252676,"kb_used_omap":7,"kb_used_meta":197432,"kb_avail":471408956,"statfs":{"total":483183820800,"available":482722770944,"internally_reserved":0,"allocated":258740224,"data_stored":445859634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7973,"internal_metadata":202170587},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11614759,"num_objects":75,"num_object_clones":23,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":18,"num_read":11086,"num_read_kb":54500,"num_write":692,"num_write_kb":8010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1490503,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.013833"},"pg_stats":[{"pgid":"3.1f","version":"47'90","reported_seq":1260,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108129+0000","last_change":"2024-09-17T21:46:25.785411+0000","last_active":"2024-09-17T21:46:31.108129+0000","last_peered":"2024-09-17T21:46:31.108129+0000","last_clean":"2024-09-17T21:46:31.108129+0000","last_became_active":"2024-09-17T21:46:25.711785+0000","last_became_peered":"2024-09-17T21:46:25.711785+0000","last_unstale":"2024-09-17T21:46:31.108129+0000","last_undegraded":"2024-09-17T21:46:31.108129+0000","last_fullsized":"2024-09-17T21:46:31.108129+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:08:11.929397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":18236981,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":2146,"num_read_kb":10723,"num_write":82,"num_write_kb":2965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":6680735,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"47'149","reported_seq":4550,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724927+0000","last_change":"2024-09-17T21:46:03.271053+0000","last_active":"2024-09-17T21:46:30.724927+0000","last_peered":"2024-09-17T21:46:30.724927+0000","last_clean":"2024-09-17T21:46:30.724927+0000","last_became_active":"2024-09-17T21:46:01.267064+0000","last_became_peered":"2024-09-17T21:46:01.267064+0000","last_unstale":"2024-09-17T21:46:30.724927+0000","last_undegraded":"2024-09-17T21:46:30.724927+0000","last_fullsized":"2024-09-17T21:46:30.724927+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:59:13.016540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":49006592,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":8,"num_read":4369,"num_read_kb":27584,"num_write":135,"num_write_kb":16082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"46'45","reported_seq":985,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.863488+0000","last_change":"2024-09-17T21:46:03.281927+0000","last_active":"2024-09-17T21:46:30.863488+0000","last_peered":"2024-09-17T21:46:30.863488+0000","last_clean":"2024-09-17T21:46:30.863488+0000","last_became_active":"2024-09-17T21:46:01.272354+0000","last_became_peered":"2024-09-17T21:46:01.272354+0000","last_unstale":"2024-09-17T21:46:30.863488+0000","last_undegraded":"2024-09-17T21:46:30.863488+0000","last_fullsized":"2024-09-17T21:46:30.863488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:19:42.186210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":11212288,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":883,"num_read_kb":4669,"num_write":64,"num_write_kb":3555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'70","reported_seq":2483,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108131+0000","last_change":"2024-09-17T21:46:03.282883+0000","last_active":"2024-09-17T21:46:31.108131+0000","last_peered":"2024-09-17T21:46:31.108131+0000","last_clean":"2024-09-17T21:46:31.108131+0000","last_became_active":"2024-09-17T21:46:01.270813+0000","last_became_peered":"2024-09-17T21:46:01.270813+0000","last_unstale":"2024-09-17T21:46:31.108131+0000","last_undegraded":"2024-09-17T21:46:31.108131+0000","last_fullsized":"2024-09-17T21:46:31.108131+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:08:53.860488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":26124288,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":2376,"num_read_kb":20211,"num_write":130,"num_write_kb":21387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"46'186","reported_seq":3140,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.550064+0000","last_change":"2024-09-17T21:46:28.714405+0000","last_active":"2024-09-17T21:46:29.550064+0000","last_peered":"2024-09-17T21:46:29.550064+0000","last_clean":"2024-09-17T21:46:29.550064+0000","last_became_active":"2024-09-17T21:46:28.713834+0000","last_became_peered":"2024-09-17T21:46:28.713834+0000","last_unstale":"2024-09-17T21:46:29.550064+0000","last_undegraded":"2024-09-17T21:46:29.550064+0000","last_fullsized":"2024-09-17T21:46:29.550064+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:00.307070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":39733760,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":4511,"num_read_kb":10168,"num_write":124,"num_write_kb":1776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3707392,"num_keys_recovered":27,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"46'36","reported_seq":1492,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.762958+0000","last_change":"2024-09-17T21:46:03.277187+0000","last_active":"2024-09-17T21:46:29.762958+0000","last_peered":"2024-09-17T21:46:29.762958+0000","last_clean":"2024-09-17T21:46:29.762958+0000","last_became_active":"2024-09-17T21:46:01.268588+0000","last_became_peered":"2024-09-17T21:46:01.268588+0000","last_unstale":"2024-09-17T21:46:29.762958+0000","last_undegraded":"2024-09-17T21:46:29.762958+0000","last_fullsized":"2024-09-17T21:46:29.762958+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:34:33.186514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":14389354,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":1407,"num_read_kb":9401,"num_write":39,"num_write_kb":2829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"48'120","reported_seq":1986,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737650+0000","last_change":"2024-09-17T21:46:03.278649+0000","last_active":"2024-09-17T21:46:33.737650+0000","last_peered":"2024-09-17T21:46:33.737650+0000","last_clean":"2024-09-17T21:46:33.737650+0000","last_became_active":"2024-09-17T21:46:01.267429+0000","last_became_peered":"2024-09-17T21:46:01.267429+0000","last_unstale":"2024-09-17T21:46:33.737650+0000","last_undegraded":"2024-09-17T21:46:33.737650+0000","last_fullsized":"2024-09-17T21:46:33.737650+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:01.981707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":4857344,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1810,"num_read_kb":7110,"num_write":118,"num_write_kb":1691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"47'123","reported_seq":2349,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.848177+0000","last_change":"2024-09-17T21:46:28.714495+0000","last_active":"2024-09-17T21:46:30.848177+0000","last_peered":"2024-09-17T21:46:30.848177+0000","last_clean":"2024-09-17T21:46:30.848177+0000","last_became_active":"2024-09-17T21:46:28.713898+0000","last_became_peered":"2024-09-17T21:46:28.713898+0000","last_unstale":"2024-09-17T21:46:30.848177+0000","last_undegraded":"2024-09-17T21:46:30.848177+0000","last_fullsized":"2024-09-17T21:46:30.848177+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:36.307253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":34762240,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":2318,"num_read_kb":4174,"num_write":106,"num_write_kb":1858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":1621504,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"46'78","reported_seq":860,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108495+0000","last_change":"2024-09-17T21:46:25.754859+0000","last_active":"2024-09-17T21:46:31.108495+0000","last_peered":"2024-09-17T21:46:31.108495+0000","last_clean":"2024-09-17T21:46:31.108495+0000","last_became_active":"2024-09-17T21:46:25.709818+0000","last_became_peered":"2024-09-17T21:46:25.709818+0000","last_unstale":"2024-09-17T21:46:31.108495+0000","last_undegraded":"2024-09-17T21:46:31.108495+0000","last_fullsized":"2024-09-17T21:46:31.108495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:03:52.086875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":5029888,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":913,"num_read_kb":4444,"num_write":95,"num_write_kb":2566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":2538496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.16","version":"46'39","reported_seq":500,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106630+0000","last_change":"2024-09-17T21:46:03.282056+0000","last_active":"2024-09-17T21:46:31.106630+0000","last_peered":"2024-09-17T21:46:31.106630+0000","last_clean":"2024-09-17T21:46:31.106630+0000","last_became_active":"2024-09-17T21:46:01.267932+0000","last_became_peered":"2024-09-17T21:46:01.267932+0000","last_unstale":"2024-09-17T21:46:31.106630+0000","last_undegraded":"2024-09-17T21:46:31.106630+0000","last_fullsized":"2024-09-17T21:46:31.106630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:33.448348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":4932608,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":393,"num_read_kb":1492,"num_write":60,"num_write_kb":2763,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"47'150","reported_seq":2743,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.061347+0000","last_change":"2024-09-17T21:46:25.826556+0000","last_active":"2024-09-17T21:46:31.061347+0000","last_peered":"2024-09-17T21:46:31.061347+0000","last_clean":"2024-09-17T21:46:31.061347+0000","last_became_active":"2024-09-17T21:46:25.716440+0000","last_became_peered":"2024-09-17T21:46:25.716440+0000","last_unstale":"2024-09-17T21:46:31.061347+0000","last_undegraded":"2024-09-17T21:46:31.061347+0000","last_fullsized":"2024-09-17T21:46:31.061347+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:49.482394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":2616832,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":2615,"num_read_kb":5156,"num_write":122,"num_write_kb":1079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":1094656,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"47'68","reported_seq":1913,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724609+0000","last_change":"2024-09-17T21:46:03.271246+0000","last_active":"2024-09-17T21:46:30.724609+0000","last_peered":"2024-09-17T21:46:30.724609+0000","last_clean":"2024-09-17T21:46:30.724609+0000","last_became_active":"2024-09-17T21:46:01.266638+0000","last_became_peered":"2024-09-17T21:46:01.266638+0000","last_unstale":"2024-09-17T21:46:30.724609+0000","last_undegraded":"2024-09-17T21:46:30.724609+0000","last_fullsized":"2024-09-17T21:46:30.724609+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:50:18.206664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":17264656,"num_objects":16,"num_object_clones":2,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":3,"num_read":1800,"num_read_kb":9798,"num_write":73,"num_write_kb":2206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724612+0000","last_change":"2024-09-17T21:45:54.626245+0000","last_active":"2024-09-17T21:46:30.724612+0000","last_peered":"2024-09-17T21:46:30.724612+0000","last_clean":"2024-09-17T21:46:30.724612+0000","last_became_active":"2024-09-17T21:45:52.269500+0000","last_became_peered":"2024-09-17T21:45:52.269500+0000","last_unstale":"2024-09-17T21:46:30.724612+0000","last_undegraded":"2024-09-17T21:46:30.724612+0000","last_fullsized":"2024-09-17T21:46:30.724612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.259001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"48'87","reported_seq":1855,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.916766+0000","last_change":"2024-09-17T21:46:25.802846+0000","last_active":"2024-09-17T21:46:33.916766+0000","last_peered":"2024-09-17T21:46:33.916766+0000","last_clean":"2024-09-17T21:46:33.916766+0000","last_became_active":"2024-09-17T21:46:25.713238+0000","last_became_peered":"2024-09-17T21:46:25.713238+0000","last_unstale":"2024-09-17T21:46:33.916766+0000","last_undegraded":"2024-09-17T21:46:33.916766+0000","last_fullsized":"2024-09-17T21:46:33.916766+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:45.219111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":11810855,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":1523,"num_read_kb":11744,"num_write":120,"num_write_kb":4502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":3456117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":81,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737904+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:46:33.737904+0000","last_peered":"2024-09-17T21:46:33.737904+0000","last_clean":"2024-09-17T21:46:33.737904+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:46:33.737904+0000","last_undegraded":"2024-09-17T21:46:33.737904+0000","last_fullsized":"2024-09-17T21:46:33.737904+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"47'80","reported_seq":1430,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.861951+0000","last_change":"2024-09-17T21:46:03.270712+0000","last_active":"2024-09-17T21:46:30.861951+0000","last_peered":"2024-09-17T21:46:30.861951+0000","last_clean":"2024-09-17T21:46:30.861951+0000","last_became_active":"2024-09-17T21:46:01.265693+0000","last_became_peered":"2024-09-17T21:46:01.265693+0000","last_unstale":"2024-09-17T21:46:30.861951+0000","last_undegraded":"2024-09-17T21:46:30.861951+0000","last_fullsized":"2024-09-17T21:46:30.861951+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:10.796820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":3275776,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":1291,"num_read_kb":5510,"num_write":75,"num_write_kb":694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724649+0000","last_change":"2024-09-17T21:46:25.714843+0000","last_active":"2024-09-17T21:46:30.724649+0000","last_peered":"2024-09-17T21:46:30.724649+0000","last_clean":"2024-09-17T21:46:30.724649+0000","last_became_active":"2024-09-17T21:46:25.714521+0000","last_became_peered":"2024-09-17T21:46:25.714521+0000","last_unstale":"2024-09-17T21:46:30.724649+0000","last_undegraded":"2024-09-17T21:46:30.724649+0000","last_fullsized":"2024-09-17T21:46:30.724649+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:05.250233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"48'78","reported_seq":2513,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737911+0000","last_change":"2024-09-17T21:46:03.279109+0000","last_active":"2024-09-17T21:46:33.737911+0000","last_peered":"2024-09-17T21:46:33.737911+0000","last_clean":"2024-09-17T21:46:33.737911+0000","last_became_active":"2024-09-17T21:46:01.268063+0000","last_became_peered":"2024-09-17T21:46:01.268063+0000","last_unstale":"2024-09-17T21:46:33.737911+0000","last_undegraded":"2024-09-17T21:46:33.737911+0000","last_fullsized":"2024-09-17T21:46:33.737911+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:16:50.614943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":6665216,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":2369,"num_read_kb":11942,"num_write":72,"num_write_kb":1416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":96,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.738193+0000","last_change":"2024-09-17T21:46:25.712667+0000","last_active":"2024-09-17T21:46:33.738193+0000","last_peered":"2024-09-17T21:46:33.738193+0000","last_clean":"2024-09-17T21:46:33.738193+0000","last_became_active":"2024-09-17T21:46:25.712313+0000","last_became_peered":"2024-09-17T21:46:25.712313+0000","last_unstale":"2024-09-17T21:46:33.738193+0000","last_undegraded":"2024-09-17T21:46:33.738193+0000","last_fullsized":"2024-09-17T21:46:33.738193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:06.825437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"47'69","reported_seq":952,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.095085+0000","last_change":"2024-09-17T21:46:03.282656+0000","last_active":"2024-09-17T21:46:31.095085+0000","last_peered":"2024-09-17T21:46:31.095085+0000","last_clean":"2024-09-17T21:46:31.095085+0000","last_became_active":"2024-09-17T21:46:01.271756+0000","last_became_peered":"2024-09-17T21:46:01.271756+0000","last_unstale":"2024-09-17T21:46:31.095085+0000","last_undegraded":"2024-09-17T21:46:31.095085+0000","last_fullsized":"2024-09-17T21:46:31.095085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:43:36.282504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":1556512,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":832,"num_read_kb":4097,"num_write":70,"num_write_kb":698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"46'86","reported_seq":1082,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.636993+0000","last_change":"2024-09-17T21:46:25.744899+0000","last_active":"2024-09-17T21:46:29.636993+0000","last_peered":"2024-09-17T21:46:29.636993+0000","last_clean":"2024-09-17T21:46:29.636993+0000","last_became_active":"2024-09-17T21:46:25.713298+0000","last_became_peered":"2024-09-17T21:46:25.713298+0000","last_unstale":"2024-09-17T21:46:29.636993+0000","last_undegraded":"2024-09-17T21:46:29.636993+0000","last_fullsized":"2024-09-17T21:46:29.636993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:00:56.686749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":2912256,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":890,"num_read_kb":1261,"num_write":74,"num_write_kb":277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":60416,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":77,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.729850+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:46:30.729850+0000","last_peered":"2024-09-17T21:46:30.729850+0000","last_clean":"2024-09-17T21:46:30.729850+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:46:30.729850+0000","last_undegraded":"2024-09-17T21:46:30.729850+0000","last_fullsized":"2024-09-17T21:46:30.729850+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"48'105","reported_seq":2122,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.917661+0000","last_change":"2024-09-17T21:46:03.278873+0000","last_active":"2024-09-17T21:46:33.917661+0000","last_peered":"2024-09-17T21:46:33.917661+0000","last_clean":"2024-09-17T21:46:33.917661+0000","last_became_active":"2024-09-17T21:46:01.266414+0000","last_became_peered":"2024-09-17T21:46:01.266414+0000","last_unstale":"2024-09-17T21:46:33.917661+0000","last_undegraded":"2024-09-17T21:46:33.917661+0000","last_fullsized":"2024-09-17T21:46:33.917661+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:02.782507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":15669371,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":2801,"num_read_kb":6611,"num_write":102,"num_write_kb":1421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.725191+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:46:30.725191+0000","last_peered":"2024-09-17T21:46:30.725191+0000","last_clean":"2024-09-17T21:46:30.725191+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:46:30.725191+0000","last_undegraded":"2024-09-17T21:46:30.725191+0000","last_fullsized":"2024-09-17T21:46:30.725191+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"46'1021","reported_seq":1818,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.800543+0000","last_change":"2024-09-17T21:46:28.714716+0000","last_active":"2024-09-17T21:46:29.800543+0000","last_peered":"2024-09-17T21:46:29.800543+0000","last_clean":"2024-09-17T21:46:29.800543+0000","last_became_active":"2024-09-17T21:46:28.713879+0000","last_became_peered":"2024-09-17T21:46:28.713879+0000","last_unstale":"2024-09-17T21:46:29.800543+0000","last_undegraded":"2024-09-17T21:46:29.800543+0000","last_fullsized":"2024-09-17T21:46:29.800543+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1021,"log_dups_size":0,"ondisk_log_size":1021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:12.313725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":3885823,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":6173,"num_read_kb":5971,"num_write":1795,"num_write_kb":2161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":98560,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724256+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:46:30.724256+0000","last_peered":"2024-09-17T21:46:30.724256+0000","last_clean":"2024-09-17T21:46:30.724256+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:46:30.724256+0000","last_undegraded":"2024-09-17T21:46:30.724256+0000","last_fullsized":"2024-09-17T21:46:30.724256+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"48'78","reported_seq":2372,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:33.737975+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:46:33.737975+0000","last_peered":"2024-09-17T21:46:33.737975+0000","last_clean":"2024-09-17T21:46:33.737975+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:46:33.737975+0000","last_undegraded":"2024-09-17T21:46:33.737975+0000","last_fullsized":"2024-09-17T21:46:33.737975+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:36:11.430369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":31046672,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2241,"num_read_kb":20787,"num_write":73,"num_write_kb":16521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.729846+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:46:30.729846+0000","last_peered":"2024-09-17T21:46:30.729846+0000","last_clean":"2024-09-17T21:46:30.729846+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:46:30.729846+0000","last_undegraded":"2024-09-17T21:46:30.729846+0000","last_fullsized":"2024-09-17T21:46:30.729846+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724259+0000","last_change":"2024-09-17T21:46:28.714299+0000","last_active":"2024-09-17T21:46:30.724259+0000","last_peered":"2024-09-17T21:46:30.724259+0000","last_clean":"2024-09-17T21:46:30.724259+0000","last_became_active":"2024-09-17T21:46:28.713856+0000","last_became_peered":"2024-09-17T21:46:28.713856+0000","last_unstale":"2024-09-17T21:46:30.724259+0000","last_undegraded":"2024-09-17T21:46:30.724259+0000","last_fullsized":"2024-09-17T21:46:30.724259+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:14.453880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"47'85","reported_seq":2229,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.861982+0000","last_change":"2024-09-17T21:46:03.270987+0000","last_active":"2024-09-17T21:46:30.861982+0000","last_peered":"2024-09-17T21:46:30.861982+0000","last_clean":"2024-09-17T21:46:30.861982+0000","last_became_active":"2024-09-17T21:46:01.270323+0000","last_became_peered":"2024-09-17T21:46:01.270323+0000","last_unstale":"2024-09-17T21:46:30.861982+0000","last_undegraded":"2024-09-17T21:46:30.861982+0000","last_fullsized":"2024-09-17T21:46:30.861982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:24:46.882120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":15392784,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":2091,"num_read_kb":9919,"num_write":80,"num_write_kb":694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"46'117","reported_seq":2177,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-17T21:46:29.774735+0000","last_change":"2024-09-17T21:46:03.276671+0000","last_active":"2024-09-17T21:46:29.774735+0000","last_peered":"2024-09-17T21:46:29.774735+0000","last_clean":"2024-09-17T21:46:29.774735+0000","last_became_active":"2024-09-17T21:46:01.263881+0000","last_became_peered":"2024-09-17T21:46:01.263881+0000","last_unstale":"2024-09-17T21:46:29.774735+0000","last_undegraded":"2024-09-17T21:46:29.774735+0000","last_fullsized":"2024-09-17T21:46:29.774735+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:04:45.049396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":34484260,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":5,"num_read":2242,"num_read_kb":19651,"num_write":166,"num_write_kb":19163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'73","reported_seq":1457,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106992+0000","last_change":"2024-09-17T21:46:03.271833+0000","last_active":"2024-09-17T21:46:31.106992+0000","last_peered":"2024-09-17T21:46:31.106992+0000","last_clean":"2024-09-17T21:46:31.106992+0000","last_became_active":"2024-09-17T21:46:01.268770+0000","last_became_peered":"2024-09-17T21:46:01.268770+0000","last_unstale":"2024-09-17T21:46:31.106992+0000","last_undegraded":"2024-09-17T21:46:31.106992+0000","last_fullsized":"2024-09-17T21:46:31.106992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:44:04.211419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":3338240,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":3,"num_read":1330,"num_read_kb":4321,"num_write":70,"num_write_kb":165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"48'104","reported_seq":2673,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T21:46:34.087171+0000","last_change":"2024-09-17T21:46:25.763286+0000","last_active":"2024-09-17T21:46:34.087171+0000","last_peered":"2024-09-17T21:46:34.087171+0000","last_clean":"2024-09-17T21:46:34.087171+0000","last_became_active":"2024-09-17T21:46:25.712470+0000","last_became_peered":"2024-09-17T21:46:25.712470+0000","last_unstale":"2024-09-17T21:46:34.087171+0000","last_undegraded":"2024-09-17T21:46:34.087171+0000","last_fullsized":"2024-09-17T21:46:34.087171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:21.259438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":11227152,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":6,"num_read":2563,"num_read_kb":13195,"num_write":118,"num_write_kb":3358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":2535440,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"47'122","reported_seq":1559,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.730003+0000","last_change":"2024-09-17T21:46:28.714996+0000","last_active":"2024-09-17T21:46:30.730003+0000","last_peered":"2024-09-17T21:46:30.730003+0000","last_clean":"2024-09-17T21:46:30.730003+0000","last_became_active":"2024-09-17T21:46:28.714320+0000","last_became_peered":"2024-09-17T21:46:28.714320+0000","last_unstale":"2024-09-17T21:46:30.730003+0000","last_undegraded":"2024-09-17T21:46:30.730003+0000","last_fullsized":"2024-09-17T21:46:30.730003+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:28:21.360651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":13533223,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1309,"num_read_kb":6849,"num_write":141,"num_write_kb":450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":386165,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'258","reported_seq":4862,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.107473+0000","last_change":"2024-09-17T21:46:03.272242+0000","last_active":"2024-09-17T21:46:31.107473+0000","last_peered":"2024-09-17T21:46:31.107473+0000","last_clean":"2024-09-17T21:46:31.107473+0000","last_became_active":"2024-09-17T21:46:01.268272+0000","last_became_peered":"2024-09-17T21:46:01.268272+0000","last_unstale":"2024-09-17T21:46:31.107473+0000","last_undegraded":"2024-09-17T21:46:31.107473+0000","last_fullsized":"2024-09-17T21:46:31.107473+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":258,"log_dups_size":0,"ondisk_log_size":258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:40.281908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":14970918,"num_objects":13,"num_object_clones":3,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":4763,"num_read_kb":7739,"num_write":232,"num_write_kb":3959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"47'55","reported_seq":373,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.829612+0000","last_change":"2024-09-17T21:46:03.283005+0000","last_active":"2024-09-17T21:46:30.829612+0000","last_peered":"2024-09-17T21:46:30.829612+0000","last_clean":"2024-09-17T21:46:30.829612+0000","last_became_active":"2024-09-17T21:46:01.272460+0000","last_became_peered":"2024-09-17T21:46:01.272460+0000","last_unstale":"2024-09-17T21:46:30.829612+0000","last_undegraded":"2024-09-17T21:46:30.829612+0000","last_fullsized":"2024-09-17T21:46:30.829612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:29:01.332286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":4431872,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":252,"num_read_kb":932,"num_write":55,"num_write_kb":455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"47'282","reported_seq":3625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.106725+0000","last_change":"2024-09-17T21:46:03.282498+0000","last_active":"2024-09-17T21:46:31.106725+0000","last_peered":"2024-09-17T21:46:31.106725+0000","last_clean":"2024-09-17T21:46:31.106725+0000","last_became_active":"2024-09-17T21:46:01.268915+0000","last_became_peered":"2024-09-17T21:46:01.268915+0000","last_unstale":"2024-09-17T21:46:31.106725+0000","last_undegraded":"2024-09-17T21:46:31.106725+0000","last_fullsized":"2024-09-17T21:46:31.106725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:22:24.434413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":28082762,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":4364,"num_read_kb":27601,"num_write":501,"num_write_kb":23495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"47'61","reported_seq":2935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108260+0000","last_change":"2024-09-17T21:46:25.778540+0000","last_active":"2024-09-17T21:46:31.108260+0000","last_peered":"2024-09-17T21:46:31.108260+0000","last_clean":"2024-09-17T21:46:31.108260+0000","last_became_active":"2024-09-17T21:46:25.711546+0000","last_became_peered":"2024-09-17T21:46:25.711546+0000","last_unstale":"2024-09-17T21:46:31.108260+0000","last_undegraded":"2024-09-17T21:46:31.108260+0000","last_fullsized":"2024-09-17T21:46:31.108260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:54.073920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":20673145,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":2616,"num_read_kb":10370,"num_write":69,"num_write_kb":1195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":883242,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"47'90","reported_seq":1369,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.828655+0000","last_change":"2024-09-17T21:46:03.271231+0000","last_active":"2024-09-17T21:46:30.828655+0000","last_peered":"2024-09-17T21:46:30.828655+0000","last_clean":"2024-09-17T21:46:30.828655+0000","last_became_active":"2024-09-17T21:46:01.266566+0000","last_became_peered":"2024-09-17T21:46:01.266566+0000","last_unstale":"2024-09-17T21:46:30.828655+0000","last_undegraded":"2024-09-17T21:46:30.828655+0000","last_fullsized":"2024-09-17T21:46:30.828655+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:56.880646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":19168256,"num_objects":14,"num_object_clones":4,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":4,"num_read":1226,"num_read_kb":15767,"num_write":87,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"46'51","reported_seq":313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:30.724349+0000","last_change":"2024-09-17T21:46:25.734932+0000","last_active":"2024-09-17T21:46:30.724349+0000","last_peered":"2024-09-17T21:46:30.724349+0000","last_clean":"2024-09-17T21:46:30.724349+0000","last_became_active":"2024-09-17T21:46:25.710701+0000","last_became_peered":"2024-09-17T21:46:25.710701+0000","last_unstale":"2024-09-17T21:46:30.724349+0000","last_undegraded":"2024-09-17T21:46:30.724349+0000","last_fullsized":"2024-09-17T21:46:30.724349+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:20:41.751095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":10022912,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":4,"num_read":160,"num_read_kb":817,"num_write":47,"num_write_kb":226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":8192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"46'97","reported_seq":1475,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T21:46:31.108787+0000","last_change":"2024-09-17T21:46:03.272109+0000","last_active":"2024-09-17T21:46:31.108787+0000","last_peered":"2024-09-17T21:46:31.108787+0000","last_clean":"2024-09-17T21:46:31.108787+0000","last_became_active":"2024-09-17T21:46:01.267697+0000","last_became_peered":"2024-09-17T21:46:01.267697+0000","last_unstale":"2024-09-17T21:46:31.108787+0000","last_undegraded":"2024-09-17T21:46:31.108787+0000","last_fullsized":"2024-09-17T21:46:31.108787+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:38.773233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":25896121,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":4,"num_read":1501,"num_read_kb":5344,"num_write":142,"num_write_kb":2940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":506210957,"num_objects":443,"num_object_clones":79,"num_object_copies":886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":443,"num_whiteouts":107,"num_read":68078,"num_read_kb":305358,"num_write":5237,"num_write_kb":159691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":23070915,"num_keys_recovered":158,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":257904640,"data_stored":446764128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4153,"ondisk_log_size":4153,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607565,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":95864,"kb_used_data":60520,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94275976,"statfs":{"total":96636764160,"available":96538599424,"internally_reserved":0,"allocated":61972480,"data_stored":114512194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607564,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99248,"kb_used_data":59744,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94272592,"statfs":{"total":96636764160,"available":96535134208,"internally_reserved":0,"allocated":61177856,"data_stored":107276640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607564,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":83416,"kb_used_data":39720,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94288424,"statfs":{"total":96636764160,"available":96551346176,"internally_reserved":0,"allocated":40673280,"data_stored":68831838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":0,"seq":51539607564,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607564,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":99324,"kb_used_data":59816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94272516,"statfs":{"total":96636764160,"available":96535056384,"internally_reserved":0,"allocated":61251584,"data_stored":106780538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72392,"kb_used_data":32876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94299448,"statfs":{"total":96636764160,"available":96562634752,"internally_reserved":0,"allocated":33665024,"data_stored":48458424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2244608,"data_stored":2244390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32505856,"data_stored":47648181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40148992,"data_stored":68627481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":60080128,"data_stored":106560965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":61460480,"data_stored":107304404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":61464576,"data_stored":114378707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:46:35.514 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:46:35.514 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:35.514 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T21:46:35.514 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:46:35.514 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:46:35.514 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T21:46:35.517 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:35.515+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.768679922, 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:35.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:11465 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:7 [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.575930064000001, 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074631050000000003, 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11465 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:35.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002674+0000", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002681+0000", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9399999999999996e-06 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002690+0000", 2024-09-17T21:46:35.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5639999999999997e-06 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.002711+0000", 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0899000000000001e-05 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.009992+0000", 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072811359999999997 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010037+0000", 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4721000000000001e-05 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:17.010137+0000", 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100447 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.759028958, 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:46:35.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:46:35.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:910 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7583037470000003, 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082095099999999997, 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 910 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820301+0000", 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820306+0000", 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7180000000000004e-06 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820601+0000", 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029544599999999999 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820609+0000", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821192+0000", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058298099999999999 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821301+0000", 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109826 2024-09-17T21:46:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.828510+0000", 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072084130000000003 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2002 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832169+0000", 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7464352999999999, 2024-09-17T21:46:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073592359999999999, 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2002 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832169+0000", 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832169+0000", 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832169+0000", 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832172+0000", 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9210000000000001e-06 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832179+0000", 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6769999999999998e-06 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832184+0000", 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1880000000000001e-06 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839260+0000", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070763029999999999 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839350+0000", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0051999999999998e-05 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839528+0000", 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00017758499999999999 2024-09-17T21:46:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2005 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7464123300000001, 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076004749999999998, 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2005 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832195+0000", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.339e-06 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832201+0000", 2024-09-17T21:46:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832206+0000", 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3090000000000003e-06 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839559+0000", 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073530169999999999 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839641+0000", 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1210000000000003e-05 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839792+0000", 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015160099999999999 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2006 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7463363640000003, 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083420109999999999, 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2006 2024-09-17T21:46:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832272+0000", 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832281+0000", 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2909999999999996e-06 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832288+0000", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840364+0000", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080765279999999995 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840441+0000", 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7028999999999997e-05 2024-09-17T21:46:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840610+0000", 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016856299999999999 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2032 3.8 3:1644799a:::rbd_data.11121f8e87bb.0000000000000017:8 [read 282624~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832422+0000", 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7461816739999998, 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075719519999999998, 2024-09-17T21:46:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2032 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832422+0000", 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832422+0000", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832423+0000", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6699999999999999e-07 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832425+0000", 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:46:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832431+0000", 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832436+0000", 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8010000000000003e-06 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839820+0000", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073837030000000001 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839902+0000", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2583000000000005e-05 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839994+0000", 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1996999999999993e-05 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2033 3.8 3:109bf8c5:::rbd_data.11121f8e87bb.000000000000001c:8 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.7461606089999999, 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077497670000000003, 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2033 2024-09-17T21:46:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832444+0000", 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832446+0000", 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832453+0000", 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832457+0000", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840020+0000", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075630999999999997 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840101+0000", 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0913000000000003e-05 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840193+0000", 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1823000000000005e-05 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.6750830890000001, 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:46:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:46:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:46:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.673620954, 2024-09-17T21:46:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:46:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.6710098360000001, 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:46:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:46:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4125 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.6702818499999998, 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098331159999999994, 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4125 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908323+0000", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908327+0000", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908334+0000", 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4880000000000001e-06 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908342+0000", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8909999999999995e-06 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914269+0000", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059269379999999997 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914325+0000", 2024-09-17T21:46:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6508000000000002e-05 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914456+0000", 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130889 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918056+0000", 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035999629999999999 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918075+0000", 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8549000000000001e-05 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918119+0000", 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4535e-05 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918155+0000", 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5954000000000002e-05 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4518 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.934933+0000", 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.643670653, 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074698669999999998, 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4518 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.934933+0000", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.934933+0000", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.934934+0000", 2024-09-17T21:46:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.935737+0000", 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080277799999999998 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.935748+0000", 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027e-05 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.935753+0000", 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2000000000000002e-06 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.935813+0000", 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0541000000000002e-05 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.935895+0000", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2021999999999995e-05 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.936379+0000", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00048397199999999999 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.942264+0000", 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0058845210000000002 2024-09-17T21:46:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.942283+0000", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9474999999999999e-05 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.942354+0000", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0716999999999994e-05 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.942403+0000", 2024-09-17T21:46:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9342000000000003e-05 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.606895379, 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:46:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.6061671569999998, 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:46:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:46:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:46:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.2725900210000001, 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:46:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:1911 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:b [read 884736~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.851303+0000", 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.72730135500000004, 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073309689999999997, 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1911 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851303+0000", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851303+0000", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851303+0000", 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899999999999995e-07 2024-09-17T21:46:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851307+0000", 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.022e-06 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851320+0000", 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3198e-05 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.851326+0000", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2419999999999997e-06 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.858387+0000", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070612519999999996 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.858517+0000", 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012982299999999999 2024-09-17T21:46:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.858634+0000", 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116673 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.70583151899999996, 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:46:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:46:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:46:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.69830937699999995, 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:46:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:46:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:35.688 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:35.690 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:35.690 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "" 2024-09-17T21:46:35.690 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:35.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:35.691 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:35.687+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:35.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:46:35.790 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:35.787+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:35.794 INFO:teuthology.orchestra.run.smithi165.stdout:1029 write 0x2218600 thru 0x2227ce1 (0xf6e2 bytes) 2024-09-17T21:46:35.797 INFO:teuthology.orchestra.run.smithi165.stdout:1031 read 0x5a34800 thru 0x5a40583 (0xbd84 bytes) 2024-09-17T21:46:35.826 INFO:teuthology.orchestra.run.smithi165.stdout:1032 write 0xe0b0e00 thru 0xe0b7fab (0x71ac bytes) 2024-09-17T21:46:35.828 INFO:teuthology.orchestra.run.smithi165.stdout:1035 write 0x5087a00 thru 0x508fd79 (0x837a bytes) 2024-09-17T21:46:35.829 INFO:teuthology.orchestra.run.smithi165.stdout:1037 read 0xde89a00 thru 0xde8beae (0x24af bytes) 2024-09-17T21:46:35.829 INFO:teuthology.orchestra.run.smithi165.stdout:1039 read 0x2297000 thru 0x2298dfb (0x1dfc bytes) 2024-09-17T21:46:35.836 INFO:teuthology.orchestra.run.smithi165.stdout:1041 trunc from 0xe0b8000 to 0x8885c00 2024-09-17T21:46:35.851 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:35.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:305 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846791+0000", 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00086093499999999996, 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000884598, 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 305 2024-09-17T21:46:35.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846791+0000", 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846791+0000", 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846792+0000", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5000000000000002e-07 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846797+0000", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5689999999999998e-06 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846815+0000", 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8059000000000001e-05 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846827+0000", 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2103000000000001e-05 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:306 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846849+0000", 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00080289700000000001, 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087025700000000004, 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 306 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846849+0000", 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846849+0000", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846850+0000", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5600000000000005e-07 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846854+0000", 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8029999999999998e-06 2024-09-17T21:46:35.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846863+0000", 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8950000000000004e-06 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846870+0000", 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8310000000000003e-06 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:307 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846887+0000", 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00076555199999999996, 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087082700000000002, 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 307 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846887+0000", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846887+0000", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846887+0000", 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2500000000000005e-07 2024-09-17T21:46:35.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846891+0000", 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.129e-06 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846903+0000", 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2791e-05 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846911+0000", 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.328e-06 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:317 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846929+0000", 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000722965, 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000857564, 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.862 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 317 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846929+0000", 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846929+0000", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846930+0000", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2600000000000002e-07 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846933+0000", 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0790000000000001e-06 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846948+0000", 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5102999999999999e-05 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846955+0000", 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.528e-06 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:318 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846964+0000", 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00068795599999999998, 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085024299999999998, 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 318 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846964+0000", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846964+0000", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846965+0000", 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-07 2024-09-17T21:46:35.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846967+0000", 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3240000000000001e-06 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846975+0000", 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3590000000000002e-06 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846980+0000", 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6409999999999999e-06 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:319 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846995+0000", 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00065722800000000004, 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084826599999999997, 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 319 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846995+0000", 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846995+0000", 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846996+0000", 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4000000000000002e-07 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846998+0000", 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5940000000000001e-06 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847013+0000", 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4344e-05 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847020+0000", 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4390000000000003e-06 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:329 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847036+0000", 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00061621199999999999, 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083419899999999999, 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 329 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847036+0000", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847036+0000", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847037+0000", 2024-09-17T21:46:35.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847040+0000", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.932e-06 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847047+0000", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4900000000000003e-06 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847055+0000", 2024-09-17T21:46:35.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.362e-06 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:330 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847062+0000", 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00059027199999999998, 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083515800000000002, 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 330 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847062+0000", 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847062+0000", 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847063+0000", 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5400000000000002e-07 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847065+0000", 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847071+0000", 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.263e-06 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847076+0000", 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0270000000000003e-06 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:331 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.879 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847082+0000", 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00056996100000000001, 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000846684, 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 331 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847082+0000", 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847082+0000", 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847083+0000", 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9299999999999999e-07 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847086+0000", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7590000000000002e-06 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847092+0000", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2210000000000002e-06 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847103+0000", 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0876000000000001e-05 2024-09-17T21:46:35.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:341 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847113+0000", 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00053981600000000004, 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085491300000000001, 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 341 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847113+0000", 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847113+0000", 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847113+0000", 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847122+0000", 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9889999999999998e-06 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847129+0000", 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3799999999999996e-06 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847137+0000", 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5050000000000001e-06 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:342 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847150+0000", 2024-09-17T21:46:35.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00050199999999999995, 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084460500000000001, 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 342 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847150+0000", 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847150+0000", 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847151+0000", 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847154+0000", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7920000000000001e-06 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847161+0000", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9589999999999998e-06 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847166+0000", 2024-09-17T21:46:35.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2900000000000002e-06 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:343 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847172+0000", 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000479903, 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085005900000000001, 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.890 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 343 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847172+0000", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847172+0000", 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847173+0000", 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847175+0000", 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.418e-06 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847182+0000", 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3319999999999997e-06 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847186+0000", 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8620000000000002e-06 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:352 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.893 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847219+0000", 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00043341199999999999, 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083096299999999999, 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 352 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847219+0000", 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847219+0000", 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847220+0000", 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0920000000000001e-06 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847225+0000", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.276e-06 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847237+0000", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2082000000000001e-05 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847244+0000", 2024-09-17T21:46:35.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0729999999999999e-06 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:353 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00038716299999999999, 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081207899999999995, 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 353 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:35.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847266+0000", 2024-09-17T21:46:35.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847269+0000", 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3919999999999999e-06 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847278+0000", 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2179999999999999e-06 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.847283+0000", 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6130000000000003e-06 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:293 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:d [sparse-read 1048576~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846609+0000", 2024-09-17T21:46:35.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0010432379999999999, 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014998699999999999, 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 293 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846609+0000", 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846609+0000", 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846610+0000", 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8100000000000001e-07 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846614+0000", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1869999999999999e-06 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846625+0000", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0372e-05 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846630+0000", 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.135e-06 2024-09-17T21:46:35.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846689+0000", 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9474e-05 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846795+0000", 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010576799999999999 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:294 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846649+0000", 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.0010029590000000001, 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001493067, 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 294 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846649+0000", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846649+0000", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846650+0000", 2024-09-17T21:46:35.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1999999999999999e-07 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846653+0000", 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3340000000000002e-06 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846665+0000", 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1141999999999999e-05 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846675+0000", 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0959e-05 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:295 3.d 3.3ba11d0d (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.846691+0000", 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00096163200000000002, 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00147893, 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:35.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 295 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846691+0000", 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846691+0000", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846691+0000", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846694+0000", 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6840000000000001e-06 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846709+0000", 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4949999999999999e-05 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.846715+0000", 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9939999999999997e-06 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:35.912 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:46:35.913 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 17 2024-09-17T21:46:35.913 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:35.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:35.916 INFO:teuthology.orchestra.run.smithi165.stdout:1042 punch from 0x7228000 to 0x7231600, (0x9600 bytes) 2024-09-17T21:46:35.916 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:35.911+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:35.920 INFO:teuthology.orchestra.run.smithi165.stdout:1044 read 0x25bac00 thru 0x25c203f (0x7440 bytes) 2024-09-17T21:46:35.920 INFO:teuthology.orchestra.run.smithi165.stdout:1045 punch from 0x6bcfe00 to 0x6bd4200, (0x4400 bytes) 2024-09-17T21:46:35.920 INFO:teuthology.orchestra.run.smithi165.stdout:1046 read 0x19b9600 thru 0x19c2cb5 (0x96b6 bytes) 2024-09-17T21:46:35.939 INFO:teuthology.orchestra.run.smithi165.stdout:1047 write 0x9b7ba00 thru 0x9b856cd (0x9cce bytes) 2024-09-17T21:46:35.939 INFO:teuthology.orchestra.run.smithi165.stdout:1048 read 0x346a600 thru 0x3470011 (0x5a12 bytes) 2024-09-17T21:46:35.940 INFO:teuthology.orchestra.run.smithi165.stdout:1049 write 0x8995200 thru 0x8999fda (0x4ddb bytes) 2024-09-17T21:46:35.940 INFO:teuthology.orchestra.run.smithi165.stdout:1051 punch from 0x699ca00 to 0x699ce00, (0x400 bytes) 2024-09-17T21:46:35.941 INFO:teuthology.orchestra.run.smithi165.stdout:1053 read 0x5fcce00 thru 0x5fdbdee (0xefef bytes) 2024-09-17T21:46:35.942 INFO:teuthology.orchestra.run.smithi165.stdout:1054 read 0x7a3d000 thru 0x7a3f474 (0x2475 bytes) 2024-09-17T21:46:35.943 INFO:teuthology.orchestra.run.smithi165.stdout:1055 read 0x7dae00 thru 0x7dbc99 (0xe9a bytes) 2024-09-17T21:46:35.943 INFO:teuthology.orchestra.run.smithi165.stdout:1058 write 0x6fd9400 thru 0x6fde0b2 (0x4cb3 bytes) 2024-09-17T21:46:35.944 INFO:teuthology.orchestra.run.smithi165.stdout:1060 read 0x6aa0200 thru 0x6aa2bcb (0x29cc bytes) 2024-09-17T21:46:35.956 INFO:teuthology.orchestra.run.smithi165.stdout:1061 write 0xb0f1e00 thru 0xb0fad6f (0x8f70 bytes) 2024-09-17T21:46:35.961 INFO:teuthology.orchestra.run.smithi165.stdout:1063 read 0x6b91a00 thru 0x6b94170 (0x2771 bytes) 2024-09-17T21:46:35.962 INFO:teuthology.orchestra.run.smithi165.stdout:1064 read 0x5981600 thru 0x5991355 (0xfd56 bytes) 2024-09-17T21:46:35.965 INFO:teuthology.orchestra.run.smithi165.stdout:1065 trunc from 0xb0fae00 to 0xd509e00 2024-09-17T21:46:35.979 INFO:teuthology.orchestra.run.smithi165.stdout:1066 write 0x5af800 thru 0x5b4945 (0x5146 bytes) 2024-09-17T21:46:35.984 INFO:teuthology.orchestra.run.smithi165.stdout:1067 read 0x6f7fc00 thru 0x6f8419b (0x459c bytes) 2024-09-17T21:46:35.985 INFO:teuthology.orchestra.run.smithi165.stdout:1068 read 0x3d1ae00 thru 0x3d2253a (0x773b bytes) 2024-09-17T21:46:35.985 INFO:teuthology.orchestra.run.smithi165.stdout:1071 punch from 0xa4b4a00 to 0xa4c0200, (0xb800 bytes) 2024-09-17T21:46:35.997 INFO:teuthology.orchestra.run.smithi165.stdout:1073 write 0xdc73000 thru 0xdc78c42 (0x5c43 bytes) 2024-09-17T21:46:36.003 INFO:teuthology.orchestra.run.smithi165.stdout:1076 read 0xb63c00 thru 0xb693f0 (0x57f1 bytes) 2024-09-17T21:46:36.003 INFO:teuthology.orchestra.run.smithi165.stdout:1077 write 0x9ca8600 thru 0x9cb0519 (0x7f1a bytes) 2024-09-17T21:46:36.004 INFO:teuthology.orchestra.run.smithi165.stdout:1078 read 0x2c37600 thru 0x2c3cb81 (0x5582 bytes) 2024-09-17T21:46:36.004 INFO:teuthology.orchestra.run.smithi165.stdout:1080 write 0x4884e00 thru 0x488a47f (0x5680 bytes) 2024-09-17T21:46:36.014 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:36.012+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:36.017 INFO:teuthology.orchestra.run.smithi165.stdout:1081 write 0xed1c600 thru 0xed1ea2c (0x242d bytes) 2024-09-17T21:46:36.022 INFO:teuthology.orchestra.run.smithi165.stdout:1082 write 0x2e45600 thru 0x2e4bd9a (0x679b bytes) 2024-09-17T21:46:36.023 INFO:teuthology.orchestra.run.smithi165.stdout:1083 read 0x57f2600 thru 0x57f8f34 (0x6935 bytes) 2024-09-17T21:46:36.024 INFO:teuthology.orchestra.run.smithi165.stdout:1085 write 0x6977400 thru 0x6984d9d (0xd99e bytes) 2024-09-17T21:46:36.024 INFO:teuthology.orchestra.run.smithi165.stdout:1086 punch from 0x4baba00 to 0x4baca00, (0x1000 bytes) 2024-09-17T21:46:36.024 INFO:teuthology.orchestra.run.smithi165.stdout:1088 write 0x4f36a00 thru 0x4f3fb4d (0x914e bytes) 2024-09-17T21:46:36.025 INFO:teuthology.orchestra.run.smithi165.stdout:1092 read 0x1b18a00 thru 0x1b24ba3 (0xc1a4 bytes) 2024-09-17T21:46:36.026 INFO:teuthology.orchestra.run.smithi165.stdout:1093 punch from 0x1b4600 to 0x1c3400, (0xee00 bytes) 2024-09-17T21:46:36.026 INFO:teuthology.orchestra.run.smithi165.stdout:1094 punch from 0x9324400 to 0x932a800, (0x6400 bytes) 2024-09-17T21:46:36.026 INFO:teuthology.orchestra.run.smithi165.stdout:1095 read 0x679800 thru 0x6880a8 (0xe8a9 bytes) 2024-09-17T21:46:36.029 INFO:teuthology.orchestra.run.smithi165.stdout:1098 write 0x391c800 thru 0x39278cc (0xb0cd bytes) 2024-09-17T21:46:36.030 INFO:teuthology.orchestra.run.smithi165.stdout:1104 punch from 0x9f7c400 to 0x9f81800, (0x5400 bytes) 2024-09-17T21:46:36.030 INFO:teuthology.orchestra.run.smithi165.stdout:1105 read 0x6b8c200 thru 0x6b98d24 (0xcb25 bytes) 2024-09-17T21:46:36.030 INFO:teuthology.orchestra.run.smithi165.stdout:1106 write 0x6185c00 thru 0x619256d (0xc96e bytes) 2024-09-17T21:46:36.031 INFO:teuthology.orchestra.run.smithi165.stdout:1110 punch from 0x1cde800 to 0x1cea200, (0xba00 bytes) 2024-09-17T21:46:36.031 INFO:teuthology.orchestra.run.smithi165.stdout:1113 punch from 0xb97ac00 to 0xb986200, (0xb600 bytes) 2024-09-17T21:46:36.032 INFO:teuthology.orchestra.run.smithi165.stdout:1114 read 0xa81f400 thru 0xa828414 (0x9015 bytes) 2024-09-17T21:46:36.036 INFO:teuthology.orchestra.run.smithi165.stdout:1115 read 0x1d33c00 thru 0x1d361cb (0x25cc bytes) 2024-09-17T21:46:36.037 INFO:teuthology.orchestra.run.smithi165.stdout:1117 write 0x97ac600 thru 0x97b52c7 (0x8cc8 bytes) 2024-09-17T21:46:36.048 INFO:teuthology.orchestra.run.smithi165.stdout:1118 trunc from 0xed1ec00 to 0x5d5f000 2024-09-17T21:46:36.069 INFO:teuthology.orchestra.run.smithi165.stdout:1119 read 0x249fc00 thru 0x24afbec (0xffed bytes) 2024-09-17T21:46:36.077 INFO:teuthology.orchestra.run.smithi165.stdout:1120 clone 12 order 24 su 1048576 sc 15 2024-09-17T21:46:36.086 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:36.086 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:36.096 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:36.139 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:36.137+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:36.205 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.043232086000003, 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:36.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3822467969999996, 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:46:36.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3646754059999999, 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:36.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:46:36.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3646417130000001, 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:36.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:46:36.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3618004099999999, 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:36.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3617230989999998, 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:36.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:46:36.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3617007059999997, 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:36.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:46:36.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3616795850000001, 2024-09-17T21:46:36.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:36.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3616588409999997, 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:46:36.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:46:36.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3612441969999995, 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:46:36.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:46:36.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3612090060000002, 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:36.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3603175429999999, 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:36.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3599514739999998, 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:46:36.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3066442560000002, 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:46:36.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.272 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3066084839999998, 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:46:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.302470112, 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:36.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:46:36.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:46:36.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.2972763909999996, 2024-09-17T21:46:36.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:46:36.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:46:36.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:46:36.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.904769798, 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:46:36.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:46:36.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:46:36.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.33347006499999998, 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.295 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:46:36.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:46:36.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.32792270200000001, 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:46:36.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:46:36.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:36.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:36.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:36.308 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:36.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:36.310 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:36.307+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:36.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:36.410 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:36.408+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:36.457 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:36.457 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:36.457 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:36.457 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:36.466 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:36.480 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:36.480 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:36.497 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:36.497 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:36.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:36.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:36.530 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:36.530 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:36.546 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:36.547 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:36.550 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:36.548+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:36.564 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:36.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:36.612 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:36.610+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:36.615 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:36.615 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:36.623 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:36.714 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:36.712+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:36.745 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:46:36.775 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:36.776 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:36.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:36.815 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:36.813+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:36.855 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone11 intact 2024-09-17T21:46:36.896 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:36.897 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:36.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:36.916 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:36.914+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:37.017 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:37.014+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:37.020 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.020 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:37.117 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:37.115+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:37.178 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.179 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:37.218 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:37.216+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:37.298 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.298 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.307 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:37.319 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:37.317+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:37.363 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-17T21:46:37.415 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.416 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.420 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:37.418+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:37.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:37.522 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:37.519+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:37.579 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.579 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:37.623 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:37.621+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:37.696 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.696 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.705 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:37.733 INFO:teuthology.orchestra.run.smithi165.stdout:1123 write 0x7c84400 thru 0x7c9084a (0xc44b bytes) 2024-09-17T21:46:37.733 INFO:teuthology.orchestra.run.smithi165.stdout:1126 read 0x45d2e00 thru 0x45d9662 (0x6863 bytes) 2024-09-17T21:46:37.734 INFO:teuthology.orchestra.run.smithi165.stdout:1127 punch from 0x445dc00 to 0x446c000, (0xe400 bytes) 2024-09-17T21:46:37.734 INFO:teuthology.orchestra.run.smithi165.stdout:1129 read 0x3c97a00 thru 0x3c9c924 (0x4f25 bytes) 2024-09-17T21:46:37.745 INFO:teuthology.orchestra.run.smithi165.stdout:1131 trunc from 0x7c90a00 to 0xcb21600 2024-09-17T21:46:37.749 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:37.747+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:37.761 INFO:teuthology.orchestra.run.smithi165.stdout:1133 write 0x2104200 thru 0x2109977 (0x5778 bytes) 2024-09-17T21:46:37.768 INFO:teuthology.orchestra.run.smithi165.stdout:1138 write 0x73e9400 thru 0x73f1e5b (0x8a5c bytes) 2024-09-17T21:46:37.768 INFO:teuthology.orchestra.run.smithi165.stdout:1139 read 0x19bd800 thru 0x19c2b85 (0x5386 bytes) 2024-09-17T21:46:37.768 INFO:teuthology.orchestra.run.smithi165.stdout:1142 read 0xc61cc00 thru 0xc62b10d (0xe50e bytes) 2024-09-17T21:46:37.768 INFO:teuthology.orchestra.run.smithi165.stdout:1143 read 0x20f5400 thru 0x2103a0c (0xe60d bytes) 2024-09-17T21:46:37.769 INFO:teuthology.orchestra.run.smithi165.stdout:1144 write 0xb478000 thru 0xb480c9c (0x8c9d bytes) 2024-09-17T21:46:37.775 INFO:teuthology.orchestra.run.smithi165.stdout:1145 trunc from 0xcb21600 to 0x7768a00 2024-09-17T21:46:37.825 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:37.822+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:37.825 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.825 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:37.848 INFO:teuthology.orchestra.run.smithi165.stdout:1148 read 0x32bc400 thru 0x32c0291 (0x3e92 bytes) 2024-09-17T21:46:37.855 INFO:teuthology.orchestra.run.smithi165.stdout:1151 trunc from 0x7768a00 to 0x34cba00 2024-09-17T21:46:37.868 INFO:teuthology.orchestra.run.smithi165.stdout:1152 read 0x26e1e00 thru 0x26e9ad1 (0x7cd2 bytes) 2024-09-17T21:46:37.870 INFO:teuthology.orchestra.run.smithi165.stdout:1154 write 0x1032800 thru 0x1036daf (0x45b0 bytes) 2024-09-17T21:46:37.871 INFO:teuthology.orchestra.run.smithi165.stdout:1155 punch from 0x13b8c00 to 0x13c6e00, (0xe200 bytes) 2024-09-17T21:46:37.871 INFO:teuthology.orchestra.run.smithi165.stdout:1156 write 0x121f200 thru 0x1227b6f (0x8970 bytes) 2024-09-17T21:46:37.898 INFO:teuthology.orchestra.run.smithi165.stdout:1157 write 0xe8ccc00 thru 0xe8cdb8b (0xf8c bytes) 2024-09-17T21:46:37.900 INFO:teuthology.orchestra.run.smithi165.stdout:1158 read 0x75b6400 thru 0x75bf5ed (0x91ee bytes) 2024-09-17T21:46:37.900 INFO:teuthology.orchestra.run.smithi165.stdout:1159 read 0xd54600 thru 0xd5eba6 (0xa5a7 bytes) 2024-09-17T21:46:37.900 INFO:teuthology.orchestra.run.smithi165.stdout:1160 read 0x1b69e00 thru 0x1b75afb (0xbcfc bytes) 2024-09-17T21:46:37.901 INFO:teuthology.orchestra.run.smithi165.stdout:1161 read 0x9d3b000 thru 0x9d3f355 (0x4356 bytes) 2024-09-17T21:46:37.901 INFO:teuthology.orchestra.run.smithi165.stdout:1163 read 0xb131000 thru 0xb131257 (0x258 bytes) 2024-09-17T21:46:37.901 INFO:teuthology.orchestra.run.smithi165.stdout:1169 write 0x152ee00 thru 0x153ce86 (0xe087 bytes) 2024-09-17T21:46:37.902 INFO:teuthology.orchestra.run.smithi165.stdout:1170 read 0xafecc00 thru 0xaffc218 (0xf619 bytes) 2024-09-17T21:46:37.902 INFO:teuthology.orchestra.run.smithi165.stdout:1172 punch from 0x3c28000 to 0x3c31600, (0x9600 bytes) 2024-09-17T21:46:37.905 INFO:teuthology.orchestra.run.smithi165.stdout:1173 trunc from 0xe8cdc00 to 0xca7ca00 2024-09-17T21:46:37.917 INFO:teuthology.orchestra.run.smithi165.stdout:1177 write 0x235d600 thru 0x235f1d2 (0x1bd3 bytes) 2024-09-17T21:46:37.923 INFO:teuthology.orchestra.run.smithi165.stdout:1178 read 0x288200 thru 0x28c3a2 (0x41a3 bytes) 2024-09-17T21:46:37.924 INFO:teuthology.orchestra.run.smithi165.stdout:1179 read 0x723da00 thru 0x724115e (0x375f bytes) 2024-09-17T21:46:37.924 INFO:teuthology.orchestra.run.smithi165.stdout:1184 punch from 0x7142800 to 0x714da00, (0xb200 bytes) 2024-09-17T21:46:37.925 INFO:teuthology.orchestra.run.smithi165.stdout:1189 read 0xc11a400 thru 0xc126461 (0xc062 bytes) 2024-09-17T21:46:37.925 INFO:teuthology.orchestra.run.smithi165.stdout:1192 write 0x2de8e00 thru 0x2deade8 (0x1fe9 bytes) 2024-09-17T21:46:37.925 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:37.922+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:37.925 INFO:teuthology.orchestra.run.smithi165.stdout:1194 read 0x266a00 thru 0x276693 (0xfc94 bytes) 2024-09-17T21:46:37.926 INFO:teuthology.orchestra.run.smithi165.stdout:1195 clone 13 order 21 su 1048576 sc 7 2024-09-17T21:46:37.986 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:37.986 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:37.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:38.038 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:38.036+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:38.102 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:38.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:38.152 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:38.150+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:38.228 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:38.226+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:38.258 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:38.258 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:38.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:38.329 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:38.327+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:38.373 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:38.373 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:38.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:38.430 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:38.427+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:38.492 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:38.492 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:38.500 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:38.541 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:38.539+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:38.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:38.631 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:38.629+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:38.647 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:38.647 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:38.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:38.699 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:38.700 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:38.716 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:38.717 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:38.732 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:38.732 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:38.740 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:38.738+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:38.751 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:38.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:38.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:46:38.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:38.768 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:38.787 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:38.787 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:38.811 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:38.843 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.649536896999997, 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:38.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:38.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:38.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9885516079999999, 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:38.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9709802170000001, 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:46:38.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9709465240000004, 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:46:38.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:46:38.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9681052210000001, 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:46:38.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.96802791, 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:46:38.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:46:38.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9680055169999999, 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:46:38.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9679843960000003, 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:46:38.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:46:38.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9679636519999999, 2024-09-17T21:46:38.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9675490079999998, 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9675138170000004, 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.893 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:38.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:46:38.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9666223540000001, 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:46:38.897 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:46:38.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.966256285, 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:46:38.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.906 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9129490669999996, 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:46:38.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:46:38.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9129132950000001, 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.912 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:46:38.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:38.916 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9087749230000002, 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:46:38.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:38.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.9035812019999998, 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:46:38.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:46:38.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:46:38.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:46:38.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5110746089999996, 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:46:38.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:46:38.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:38.936 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.939774876, 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:46:38.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:38.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:46:38.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.9342275129999997, 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:38.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:38.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:46:38.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:46:38.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:46:38.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:38.949 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:38.950 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone12 from 0xca7ca00 (overlap 0x34cba00) to 0x1fad600 2024-09-17T21:46:38.951 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:38.948+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:38.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:39.051 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:39.049+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:39.101 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:39.101 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:39.101 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:39.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:39.152 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:39.150+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:39.221 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.221 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:39.221 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:39.221 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:39.221 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:39.230 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:39.273 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:39.271+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.386224031000001, 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:39.341 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:39.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:39.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.384198979000001, 2024-09-17T21:46:39.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:39.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.370212923, 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:39.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.368251716, 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:39.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:39.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.366218359000001, 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.364240441, 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:39.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:39.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:39.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.364217647, 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.368 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.372 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.358308584, 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:39.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.356277524999999, 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:39.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:39.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.356250951, 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:39.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.387 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.355641158000001, 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.388 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:39.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:39.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:39.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.355623571999999, 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:39.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:39.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:39.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.355608615000001, 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:39.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.639927127, 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:39.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:39.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.639896505999999, 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:39.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:39.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:39.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:39.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.639871888000002, 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:39.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:39.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.639849941000001, 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:39.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:39.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.639828471000001, 2024-09-17T21:46:39.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:39.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:39.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:39.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.598411668000001, 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:46:39.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:46:39.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.597367208, 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:46:39.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:46:39.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:46:39.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.447 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:39.448 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-17T21:46:39.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:39.449 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:39.446+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:39.549 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:39.546+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:39.608 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:39.608 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:39.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:39.650 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:39.647+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:39.707 INFO:teuthology.orchestra.run.smithi165.stdout:1196 punch from 0x70fe800 to 0x7107400, (0x8c00 bytes) 2024-09-17T21:46:39.716 INFO:teuthology.orchestra.run.smithi165.stdout:1197 read 0x888200 thru 0x890266 (0x8067 bytes) 2024-09-17T21:46:39.716 INFO:teuthology.orchestra.run.smithi165.stdout:1198 write 0x48e9a00 thru 0x48ee335 (0x4936 bytes) 2024-09-17T21:46:39.717 INFO:teuthology.orchestra.run.smithi165.stdout:1199 write 0x263dc00 thru 0x26423df (0x47e0 bytes) 2024-09-17T21:46:39.717 INFO:teuthology.orchestra.run.smithi165.stdout:1202 read 0x7e8600 thru 0x7f62f1 (0xdcf2 bytes) 2024-09-17T21:46:39.718 INFO:teuthology.orchestra.run.smithi165.stdout:1204 write 0x2501800 thru 0x250e04f (0xc850 bytes) 2024-09-17T21:46:39.719 INFO:teuthology.orchestra.run.smithi165.stdout:1205 punch from 0xc012800 to 0xc022400, (0xfc00 bytes) 2024-09-17T21:46:39.734 INFO:teuthology.orchestra.run.smithi165.stdout:1207 trunc from 0xca7ca00 to 0x75bee00 2024-09-17T21:46:39.748 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:46:39.750 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:39.747+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:39.763 INFO:teuthology.orchestra.run.smithi165.stdout:1208 write 0x21b0000 thru 0x21bcfcf (0xcfd0 bytes) 2024-09-17T21:46:39.764 INFO:teuthology.orchestra.run.smithi165.stdout:1209 read 0x4121e00 thru 0x412ab7e (0x8d7f bytes) 2024-09-17T21:46:39.766 INFO:teuthology.orchestra.run.smithi165.stdout:1210 read 0x2cbe800 thru 0x2cc838c (0x9b8d bytes) 2024-09-17T21:46:39.774 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.774 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:39.775 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:39.775 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:39.775 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:39.784 INFO:teuthology.orchestra.run.smithi165.stdout:1217 write 0xb5c1000 thru 0xb5c889f (0x78a0 bytes) 2024-09-17T21:46:39.784 INFO:teuthology.orchestra.run.smithi165.stdout:1218 punch from 0xa75a200 to 0xa75d800, (0x3600 bytes) 2024-09-17T21:46:39.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:39.786 INFO:teuthology.orchestra.run.smithi165.stdout:1219 read 0xa4eac00 thru 0xa4eb6b1 (0xab2 bytes) 2024-09-17T21:46:39.786 INFO:teuthology.orchestra.run.smithi165.stdout:1221 write 0x51d6400 thru 0x51d8b99 (0x279a bytes) 2024-09-17T21:46:39.788 INFO:teuthology.orchestra.run.smithi165.stdout:1222 trunc from 0xb5c8a00 to 0x9364e00 2024-09-17T21:46:39.804 INFO:teuthology.orchestra.run.smithi165.stdout:1224 read 0x8f37600 thru 0x8f402d0 (0x8cd1 bytes) 2024-09-17T21:46:39.816 INFO:teuthology.orchestra.run.smithi165.stdout:1225 write 0xbe79000 thru 0xbe80519 (0x751a bytes) 2024-09-17T21:46:39.819 INFO:teuthology.orchestra.run.smithi165.stdout:1233 trunc from 0xbe80600 to 0xd969000 2024-09-17T21:46:39.838 INFO:teuthology.orchestra.run.smithi165.stdout:1236 trunc from 0xd969000 to 0xb024800 2024-09-17T21:46:39.858 INFO:teuthology.orchestra.run.smithi165.stdout:1237 punch from 0x5443c00 to 0x5446e00, (0x3200 bytes) 2024-09-17T21:46:39.858 INFO:teuthology.orchestra.run.smithi165.stdout:1238 write 0x1287c00 thru 0x12890e4 (0x14e5 bytes) 2024-09-17T21:46:39.859 INFO:teuthology.orchestra.run.smithi165.stdout:1240 write 0x359b800 thru 0x359be82 (0x683 bytes) 2024-09-17T21:46:39.859 INFO:teuthology.orchestra.run.smithi165.stdout:1241 write 0x6be8c00 thru 0x6bf53c2 (0xc7c3 bytes) 2024-09-17T21:46:39.869 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:39.867+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:39.871 INFO:teuthology.orchestra.run.smithi165.stdout:1242 trunc from 0xb024800 to 0x151f800 2024-09-17T21:46:39.935 INFO:teuthology.orchestra.run.smithi165.stdout:1243 trunc from 0x151f800 to 0xceb1200 2024-09-17T21:46:39.949 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975603408000001, 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975580914999998, 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975556654999998, 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975530771999999, 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975505771000002, 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975465971999999, 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:39.978 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975450227, 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:39.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975438692000001, 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:39.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.975428392000001, 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:39.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973457149000001, 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:39.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:39.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973444001000001, 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:39.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:39.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:40.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973433395000001, 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.002 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:40.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973423057000002, 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:40.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:40.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973413050000001, 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:40.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973403098999999, 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:40.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.973387791, 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:40.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.971411904, 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:40.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.971400947999999, 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:40.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.971388459, 2024-09-17T21:46:40.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:40.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:40.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.971377996000001, 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:40.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:40.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:40.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:40.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:40.045 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:40.047 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:40.045+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:40.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:40.049 INFO:teuthology.orchestra.run.smithi165.stdout:1244 punch from 0x47a8800 to 0x47b2a00, (0xa200 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1246 read 0x3511200 thru 0x35143c6 (0x31c7 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1248 read 0x4dd5000 thru 0x4de1e08 (0xce09 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1249 write 0x8cbac00 thru 0x8cbbc40 (0x1041 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:skipping zero length punch hole 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1252 read 0x4e52400 thru 0x4e548ed (0x24ee bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1254 punch from 0xa849600 to 0xa84ca00, (0x3400 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1255 read 0x2e2400 thru 0x2e7506 (0x5107 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1256 read 0xc971c00 thru 0xc97c9f0 (0xadf1 bytes) 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1257 trunc from 0xceb1200 to 0x470d000 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1258 trunc from 0x470d000 to 0xd41aa00 2024-09-17T21:46:40.050 INFO:teuthology.orchestra.run.smithi165.stdout:1259 trunc from 0xd41aa00 to 0x8384800 2024-09-17T21:46:40.051 INFO:teuthology.orchestra.run.smithi165.stdout:1261 punch from 0x5b17600 to 0x5b18000, (0xa00 bytes) 2024-09-17T21:46:40.051 INFO:teuthology.orchestra.run.smithi165.stdout:1262 read 0x5804e00 thru 0x5807c76 (0x2e77 bytes) 2024-09-17T21:46:40.051 INFO:teuthology.orchestra.run.smithi165.stdout:1263 write 0xd329a00 thru 0xd32f0df (0x56e0 bytes) 2024-09-17T21:46:40.051 INFO:teuthology.orchestra.run.smithi165.stdout:1264 read 0x5942a00 thru 0x594be95 (0x9496 bytes) 2024-09-17T21:46:40.059 INFO:teuthology.orchestra.run.smithi165.stdout:1265 trunc from 0xd32f200 to 0x2cb1800 2024-09-17T21:46:40.094 INFO:teuthology.orchestra.run.smithi165.stdout:1266 write 0x3b02a00 thru 0x3b0cd42 (0xa343 bytes) 2024-09-17T21:46:40.094 INFO:teuthology.orchestra.run.smithi165.stdout:1268 write 0xfd5600 thru 0xfe5418 (0xfe19 bytes) 2024-09-17T21:46:40.108 INFO:teuthology.orchestra.run.smithi165.stdout:1270 write 0x4be0a00 thru 0x4be5a87 (0x5088 bytes) 2024-09-17T21:46:40.115 INFO:teuthology.orchestra.run.smithi165.stdout:1273 read 0x2bda000 thru 0x2bda209 (0x20a bytes) 2024-09-17T21:46:40.138 INFO:teuthology.orchestra.run.smithi165.stdout:1274 write 0xddc9200 thru 0xddcd46a (0x426b bytes) 2024-09-17T21:46:40.145 INFO:teuthology.orchestra.run.smithi165.stdout:1276 write 0x125aa00 thru 0x1269ec9 (0xf4ca bytes) 2024-09-17T21:46:40.146 INFO:teuthology.orchestra.run.smithi165.stdout:1277 write 0x7ed8c00 thru 0x7ee451c (0xb91d bytes) 2024-09-17T21:46:40.146 INFO:teuthology.orchestra.run.smithi165.stdout:1279 punch from 0xc996e00 to 0xc9a6000, (0xf200 bytes) 2024-09-17T21:46:40.147 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:40.145+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:40.148 INFO:teuthology.orchestra.run.smithi165.stdout:1285 write 0x88f7a00 thru 0x88f8c60 (0x1261 bytes) 2024-09-17T21:46:40.148 INFO:teuthology.orchestra.run.smithi165.stdout:1287 punch from 0xcb81e00 to 0xcb88000, (0x6200 bytes) 2024-09-17T21:46:40.148 INFO:teuthology.orchestra.run.smithi165.stdout:1288 read 0xcf4ac00 thru 0xcf53a6e (0x8e6f bytes) 2024-09-17T21:46:40.166 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:40.166 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:40.166 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.166 INFO:teuthology.orchestra.run.smithi165.stdout:1289 write 0xe0d9e00 thru 0xe0e6b6c (0xcd6d bytes) 2024-09-17T21:46:40.167 INFO:teuthology.orchestra.run.smithi165.stdout:1295 read 0xaf67000 thru 0xaf67614 (0x615 bytes) 2024-09-17T21:46:40.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:40.176 INFO:teuthology.orchestra.run.smithi165.stdout:1296 trunc from 0xe0e6c00 to 0x60e7e00 2024-09-17T21:46:40.213 INFO:teuthology.orchestra.run.smithi165.stdout:1299 write 0x8a6d400 thru 0x8a7a2fe (0xceff bytes) 2024-09-17T21:46:40.214 INFO:teuthology.orchestra.run.smithi165.stdout:1304 punch from 0x52c600 to 0x52fc00, (0x3600 bytes) 2024-09-17T21:46:40.219 INFO:teuthology.orchestra.run.smithi165.stdout:1305 read 0x7041600 thru 0x7048f15 (0x7916 bytes) 2024-09-17T21:46:40.219 INFO:teuthology.orchestra.run.smithi165.stdout:1308 read 0x2437e00 thru 0x2438bab (0xdac bytes) 2024-09-17T21:46:40.219 INFO:teuthology.orchestra.run.smithi165.stdout:1309 read 0x348f800 thru 0x349cf2b (0xd72c bytes) 2024-09-17T21:46:40.219 INFO:teuthology.orchestra.run.smithi165.stdout:1310 read 0x1163e00 thru 0x11642d4 (0x4d5 bytes) 2024-09-17T21:46:40.220 INFO:teuthology.orchestra.run.smithi165.stdout:1314 punch from 0x4a77200 to 0x4a7a200, (0x3000 bytes) 2024-09-17T21:46:40.220 INFO:teuthology.orchestra.run.smithi165.stdout:1315 read 0x270f800 thru 0x270fa2b (0x22c bytes) 2024-09-17T21:46:40.220 INFO:teuthology.orchestra.run.smithi165.stdout:1316 write 0x157400 thru 0x15ce64 (0x5a65 bytes) 2024-09-17T21:46:40.222 INFO:teuthology.orchestra.run.smithi165.stdout:1317 read 0x7ea0000 thru 0x7eab243 (0xb244 bytes) 2024-09-17T21:46:40.222 INFO:teuthology.orchestra.run.smithi165.stdout:1318 write 0x713ea00 thru 0x714245c (0x3a5d bytes) 2024-09-17T21:46:40.222 INFO:teuthology.orchestra.run.smithi165.stdout:1321 read 0x8734400 thru 0x874084a (0xc44b bytes) 2024-09-17T21:46:40.223 INFO:teuthology.orchestra.run.smithi165.stdout:1323 flatten 2024-09-17T21:46:40.248 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:40.245+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:40.272 INFO:teuthology.orchestra.run.smithi165.stdout:1325 write 0x8e7f600 thru 0x8e8cb6e (0xd56f bytes) 2024-09-17T21:46:40.272 INFO:teuthology.orchestra.run.smithi165.stdout:1326 read 0x6e22c00 thru 0x6e26c9c (0x409d bytes) 2024-09-17T21:46:40.277 INFO:teuthology.orchestra.run.smithi165.stdout:1327 trunc from 0x8e8cc00 to 0x4a2d000 2024-09-17T21:46:40.291 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:40.291 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:40.292 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:40.292 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.299 INFO:teuthology.orchestra.run.smithi165.stdout:1328 read 0x43d6e00 thru 0x43d7b7f (0xd80 bytes) 2024-09-17T21:46:40.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:40.301 INFO:teuthology.orchestra.run.smithi165.stdout:1329 punch from 0x318ac00 to 0x3198600, (0xda00 bytes) 2024-09-17T21:46:40.301 INFO:teuthology.orchestra.run.smithi165.stdout:1330 punch from 0x1905c00 to 0x1907a00, (0x1e00 bytes) 2024-09-17T21:46:40.312 INFO:teuthology.orchestra.run.smithi165.stdout:1333 trunc from 0x4a2d000 to 0xd9e3200 2024-09-17T21:46:40.322 INFO:teuthology.orchestra.run.smithi165.stdout:1334 trunc from 0xd9e3200 to 0xd52ec00 2024-09-17T21:46:40.332 INFO:teuthology.orchestra.run.smithi165.stdout:1335 write 0xa28000 thru 0xa29590 (0x1591 bytes) 2024-09-17T21:46:40.333 INFO:teuthology.orchestra.run.smithi165.stdout:1336 write 0x51b1400 thru 0x51c01f6 (0xedf7 bytes) 2024-09-17T21:46:40.334 INFO:teuthology.orchestra.run.smithi165.stdout:1342 read 0x1018400 thru 0x101bc80 (0x3881 bytes) 2024-09-17T21:46:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:1344 read 0xb2f6800 thru 0xb2fe8ee (0x80ef bytes) 2024-09-17T21:46:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:1345 write 0xa287200 thru 0xa296065 (0xee66 bytes) 2024-09-17T21:46:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:1346 write 0xd029400 thru 0xd02aa55 (0x1656 bytes) 2024-09-17T21:46:40.335 INFO:teuthology.orchestra.run.smithi165.stdout:1349 read 0x6707000 thru 0x67081bb (0x11bc bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1350 read 0x94eec00 thru 0x94fbc53 (0xd054 bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1351 write 0x4509800 thru 0x4516ad9 (0xd2da bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1352 read 0x821c600 thru 0x8222862 (0x6263 bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1354 write 0xc82a600 thru 0xc8333f8 (0x8df9 bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1357 read 0x9b5c200 thru 0x9b6ab14 (0xe915 bytes) 2024-09-17T21:46:40.336 INFO:teuthology.orchestra.run.smithi165.stdout:1361 punch from 0x17ccc00 to 0x17d1c00, (0x5000 bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1363 write 0x7754c00 thru 0x775b9fa (0x6dfb bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1364 write 0x89fa600 thru 0x8a075df (0xcfe0 bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1367 write 0x4d47600 thru 0x4d4aa7a (0x347b bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1370 write 0xb683000 thru 0xb68aefe (0x7eff bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1372 read 0xbc0c000 thru 0xbc1667d (0xa67e bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1373 read 0xacce200 thru 0xaccec06 (0xa07 bytes) 2024-09-17T21:46:40.337 INFO:teuthology.orchestra.run.smithi165.stdout:1375 read 0x5a59000 thru 0x5a611fd (0x81fe bytes) 2024-09-17T21:46:40.338 INFO:teuthology.orchestra.run.smithi165.stdout:1378 write 0xa295400 thru 0xa2a317a (0xdd7b bytes) 2024-09-17T21:46:40.338 INFO:teuthology.orchestra.run.smithi165.stdout:1379 write 0x16ac800 thru 0x16b8235 (0xba36 bytes) 2024-09-17T21:46:40.338 INFO:teuthology.orchestra.run.smithi165.stdout:1382 punch from 0x3efda00 to 0x3f0bc00, (0xe200 bytes) 2024-09-17T21:46:40.342 INFO:teuthology.orchestra.run.smithi165.stdout:1383 read 0xd1f4400 thru 0xd203d18 (0xf919 bytes) 2024-09-17T21:46:40.342 INFO:teuthology.orchestra.run.smithi165.stdout:1384 write 0x60c0600 thru 0x60cd49e (0xce9f bytes) 2024-09-17T21:46:40.342 INFO:teuthology.orchestra.run.smithi165.stdout:1385 read 0xcf5ae00 thru 0xcf60114 (0x5315 bytes) 2024-09-17T21:46:40.342 INFO:teuthology.orchestra.run.smithi165.stdout:1387 read 0x2ed3e00 thru 0x2edbefe (0x80ff bytes) 2024-09-17T21:46:40.342 INFO:teuthology.orchestra.run.smithi165.stdout:1389 punch from 0x8b9aa00 to 0x8ba4800, (0x9e00 bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1391 write 0x6c4ba00 thru 0x6c4e539 (0x2b3a bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1393 punch from 0x9ccfa00 to 0x9cdda00, (0xe000 bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1394 read 0x7f29c00 thru 0x7f2e4d3 (0x48d4 bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1395 read 0x8ea6e00 thru 0x8eb6032 (0xf233 bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1396 read 0x154c600 thru 0x154ee4c (0x284d bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1398 read 0x7a52400 thru 0x7a533ee (0xfef bytes) 2024-09-17T21:46:40.343 INFO:teuthology.orchestra.run.smithi165.stdout:1399 read 0x5a6000 thru 0x5aedf2 (0x8df3 bytes) 2024-09-17T21:46:40.344 INFO:teuthology.orchestra.run.smithi165.stdout:1400 write 0x5ef0a00 thru 0x5ef0af9 (0xfa bytes) 2024-09-17T21:46:40.344 INFO:teuthology.orchestra.run.smithi165.stdout:1401 read 0x70eae00 thru 0x70f2303 (0x7504 bytes) 2024-09-17T21:46:40.344 INFO:teuthology.orchestra.run.smithi165.stdout:1402 punch from 0x1b40600 to 0x1b43000, (0x2a00 bytes) 2024-09-17T21:46:40.348 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:40.346+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:40.361 INFO:teuthology.orchestra.run.smithi165.stdout:1404 write 0xdc9e600 thru 0xdcabde8 (0xd7e9 bytes) 2024-09-17T21:46:40.368 INFO:teuthology.orchestra.run.smithi165.stdout:1405 punch from 0x81e00 to 0x83e00, (0x2000 bytes) 2024-09-17T21:46:40.370 INFO:teuthology.orchestra.run.smithi165.stdout:1408 write 0x5b7e800 thru 0x5b8414e (0x594f bytes) 2024-09-17T21:46:40.370 INFO:teuthology.orchestra.run.smithi165.stdout:1409 write 0x3ac1800 thru 0x3aca74f (0x8f50 bytes) 2024-09-17T21:46:40.384 INFO:teuthology.orchestra.run.smithi165.stdout:1410 trunc from 0xdcabe00 to 0x1930600 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:40.417 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.131056024000003, 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:40.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.10912922, 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:40.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.757741184, 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:40.429 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:40.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252812066000001, 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:40.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252802330000002, 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:40.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252792718999999, 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:40.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252781888000001, 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:40.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252768296999999, 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:40.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:40.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.252757946999999, 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:40.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.24764557, 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:40.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.243318875, 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:40.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.243179666, 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:40.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:40.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:40.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.243162676000001, 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.243141678000001, 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:40.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.238295524000002, 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.238283115000002, 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.238272782999999, 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.731028887000001, 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:40.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.698753912000001, 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:40.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.512 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.698515144000002, 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:40.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:40.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.520 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1411 write 0xe7e6600 thru 0xe7f5abf (0xf4c0 bytes) 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1412 read 0x2c9ea00 thru 0x2cadae4 (0xf0e5 bytes) 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1413 write 0x7ba1800 thru 0x7baf2f0 (0xdaf1 bytes) 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1416 trunc from 0xe7f5c00 to 0x24d3000 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1417 write 0xba74c00 thru 0xba823cf (0xd7d0 bytes) 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1418 punch from 0x832200 to 0x83f200, (0xd000 bytes) 2024-09-17T21:46:40.521 INFO:teuthology.orchestra.run.smithi165.stdout:1419 trunc from 0xba82400 to 0xc89c00 2024-09-17T21:46:40.522 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:40.517+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:40.522 INFO:teuthology.orchestra.run.smithi165.stdout:1421 punch from 0x7b0a00 to 0x7b1e00, (0x1400 bytes) 2024-09-17T21:46:40.543 INFO:teuthology.orchestra.run.smithi165.stdout:1422 write 0x7576e00 thru 0x7578519 (0x171a bytes) 2024-09-17T21:46:40.543 INFO:teuthology.orchestra.run.smithi165.stdout:1423 read 0x2537800 thru 0x2541587 (0x9d88 bytes) 2024-09-17T21:46:40.543 INFO:teuthology.orchestra.run.smithi165.stdout:1424 read 0x4aac000 thru 0x4ab79d7 (0xb9d8 bytes) 2024-09-17T21:46:40.544 INFO:teuthology.orchestra.run.smithi165.stdout:1426 read 0x1612000 thru 0x16182ac (0x62ad bytes) 2024-09-17T21:46:40.561 INFO:teuthology.orchestra.run.smithi165.stdout:1427 write 0xc45c600 thru 0xc465827 (0x9228 bytes) 2024-09-17T21:46:40.562 INFO:teuthology.orchestra.run.smithi165.stdout:1430 read 0x1a1c400 thru 0x1a1f8ac (0x34ad bytes) 2024-09-17T21:46:40.562 INFO:teuthology.orchestra.run.smithi165.stdout:1432 punch from 0xb530600 to 0xb531e00, (0x1800 bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1433 read 0xb22cc00 thru 0xb23154f (0x4950 bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1435 write 0x10e7800 thru 0x10e9981 (0x2182 bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1438 write 0x9f2fe00 thru 0x9f3d705 (0xd906 bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1440 read 0x93d9a00 thru 0x93e32bc (0x98bd bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1441 read 0x4fab600 thru 0x4fac89b (0x129c bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1447 read 0x1e1cc00 thru 0x1e2a7a8 (0xdba9 bytes) 2024-09-17T21:46:40.564 INFO:teuthology.orchestra.run.smithi165.stdout:1448 read 0x20e0800 thru 0x20e6e31 (0x6632 bytes) 2024-09-17T21:46:40.565 INFO:teuthology.orchestra.run.smithi165.stdout:1450 write 0x9f42600 thru 0x9f4a2af (0x7cb0 bytes) 2024-09-17T21:46:40.565 INFO:teuthology.orchestra.run.smithi165.stdout:1452 read 0x2388600 thru 0x2394bb4 (0xc5b5 bytes) 2024-09-17T21:46:40.565 INFO:teuthology.orchestra.run.smithi165.stdout:1455 punch from 0x3a0a00 to 0x3a5600, (0x4c00 bytes) 2024-09-17T21:46:40.565 INFO:teuthology.orchestra.run.smithi165.stdout:1456 punch from 0x4a92a00 to 0x4a9b800, (0x8e00 bytes) 2024-09-17T21:46:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:1457 read 0x780ea00 thru 0x781b394 (0xc995 bytes) 2024-09-17T21:46:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:1458 read 0x6102800 thru 0x6108647 (0x5e48 bytes) 2024-09-17T21:46:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:1459 write 0x233c200 thru 0x234772a (0xb52b bytes) 2024-09-17T21:46:40.566 INFO:teuthology.orchestra.run.smithi165.stdout:1462 write 0x39dfc00 thru 0x39e3b05 (0x3f06 bytes) 2024-09-17T21:46:40.578 INFO:teuthology.orchestra.run.smithi165.stdout:1464 trunc from 0xc465a00 to 0xcfc800 2024-09-17T21:46:40.597 INFO:teuthology.orchestra.run.smithi165.stdout:1467 read 0x6f5e00 thru 0x6fd77e (0x797f bytes) 2024-09-17T21:46:40.598 INFO:teuthology.orchestra.run.smithi165.stdout:1469 read 0x9d9a00 thru 0x9e254c (0x8b4d bytes) 2024-09-17T21:46:40.598 INFO:teuthology.orchestra.run.smithi165.stdout:1473 read 0x1e1600 thru 0x1e3992 (0x2393 bytes) 2024-09-17T21:46:40.621 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:40.618+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:40.621 INFO:teuthology.orchestra.run.smithi165.stdout:1478 write 0xc020200 thru 0xc02e5f0 (0xe3f1 bytes) 2024-09-17T21:46:40.633 INFO:teuthology.orchestra.run.smithi165.stdout:1479 trunc from 0xc02e600 to 0x36cc000 2024-09-17T21:46:40.657 INFO:teuthology.orchestra.run.smithi165.stdout:1481 punch from 0x29c3000 to 0x29c6600, (0x3600 bytes) 2024-09-17T21:46:40.672 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.672 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:40.672 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:40.672 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:40.692 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:40.692 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:40.692 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T21:46:40.693 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:46:40.693 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:46:40.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T21:46:40.722 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:40.719+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:40.789 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.789 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:40.790 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:40.790 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:40.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:40.822 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:40.819+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:40.833 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.833 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:46:40.833 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:40.860 INFO:teuthology.orchestra.run.smithi165.stdout:1482 read 0x19c3800 thru 0x19cc156 (0x8957 bytes) 2024-09-17T21:46:40.860 INFO:teuthology.orchestra.run.smithi165.stdout:1483 punch from 0x21d5200 to 0x21da600, (0x5400 bytes) 2024-09-17T21:46:40.860 INFO:teuthology.orchestra.run.smithi165.stdout:1486 clone 14 order 22 su 2097152 sc 6 2024-09-17T21:46:40.912 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:40.912 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:40.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005312190000001, 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005301888999998, 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005285815000001, 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005274602, 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005257164, 2024-09-17T21:46:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:40.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005246293999999, 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:40.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:40.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:40.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.005051417000001, 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:40.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.003346746999998, 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.003336108999999, 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:40.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.003326015999999, 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:40.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.003315897, 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:40.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:40.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.001185667000001, 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:40.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.001175935999999, 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:40.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:40.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.000392577, 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:40.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:40.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.000377007000001, 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:40.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:40.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.000284104999999, 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.999789486000001, 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:40.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:40.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.999645440999998, 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.998960004000001, 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:40.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:40.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:40.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.998597614000001, 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:40.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:40.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:41.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:41.002 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:41.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:41.004 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:41.004 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:41.005 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:41.005 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:41.005 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:41.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:41.006 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:41.006 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:41.007 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:41.001+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:41.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:41.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:41.104 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:41.102+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:41.116 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.116 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:41.206 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:41.203+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:41.273 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.273 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.282 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:41.306 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:41.304+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:41.389 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.389 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:41.442 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:41.439+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:41.508 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:41.506+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.518 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:41.609 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:41.607+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:41.676 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.676 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.685 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:41.710 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:41.708+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:41.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:46:41.794 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.794 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:41.811 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:41.808+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:41.911 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:41.909+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:41.914 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:41.915 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:41.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:42.012 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:42.009+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:42.033 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.034 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.042 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:42.126 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:42.124+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:42.192 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.192 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:42.208 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-17T21:46:42.214 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:42.212+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:42.315 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:42.312+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:42.355 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.356 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:42.385 INFO:teuthology.orchestra.run.smithi165.stdout:1489 punch from 0x1bde400 to 0x1be2c00, (0x4800 bytes) 2024-09-17T21:46:42.393 INFO:teuthology.orchestra.run.smithi165.stdout:1490 read 0x1c30000 thru 0x1c3bdc3 (0xbdc4 bytes) 2024-09-17T21:46:42.393 INFO:teuthology.orchestra.run.smithi165.stdout:1494 punch from 0xede800 to 0xee1e00, (0x3600 bytes) 2024-09-17T21:46:42.416 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:42.414+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:42.437 INFO:teuthology.orchestra.run.smithi165.stdout:1495 write 0xe301c00 thru 0xe30b69d (0x9a9e bytes) 2024-09-17T21:46:42.437 INFO:teuthology.orchestra.run.smithi165.stdout:1496 read 0x2900600 thru 0x2906386 (0x5d87 bytes) 2024-09-17T21:46:42.437 INFO:teuthology.orchestra.run.smithi165.stdout:1499 read 0x7060400 thru 0x706886b (0x846c bytes) 2024-09-17T21:46:42.438 INFO:teuthology.orchestra.run.smithi165.stdout:1503 write 0xdfea000 thru 0xdfea979 (0x97a bytes) 2024-09-17T21:46:42.453 INFO:teuthology.orchestra.run.smithi165.stdout:1504 trunc from 0xe30b800 to 0xce7c00 2024-09-17T21:46:42.497 INFO:teuthology.orchestra.run.smithi165.stdout:1505 punch from 0xa98e00 to 0xaa1000, (0x8200 bytes) 2024-09-17T21:46:42.497 INFO:teuthology.orchestra.run.smithi165.stdout:1507 punch from 0x44fa00 to 0x453c00, (0x4200 bytes) 2024-09-17T21:46:42.497 INFO:teuthology.orchestra.run.smithi165.stdout:1508 read 0x466000 thru 0x46e810 (0x8811 bytes) 2024-09-17T21:46:42.499 INFO:teuthology.orchestra.run.smithi165.stdout:1509 read 0x4ec00 thru 0x5d19f (0xe5a0 bytes) 2024-09-17T21:46:42.499 INFO:teuthology.orchestra.run.smithi165.stdout:1511 punch from 0x198400 to 0x1a2000, (0x9c00 bytes) 2024-09-17T21:46:42.517 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:42.515+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:42.517 INFO:teuthology.orchestra.run.smithi165.stdout:1513 write 0x3697a00 thru 0x369f7a3 (0x7da4 bytes) 2024-09-17T21:46:42.517 INFO:teuthology.orchestra.run.smithi165.stdout:1514 read 0x357a00 thru 0x35e345 (0x6946 bytes) 2024-09-17T21:46:42.522 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.522 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:42.538 INFO:teuthology.orchestra.run.smithi165.stdout:1515 write 0x4eff200 thru 0x4f065a2 (0x73a3 bytes) 2024-09-17T21:46:42.550 INFO:teuthology.orchestra.run.smithi165.stdout:1518 write 0x4f77c00 thru 0x4f7941f (0x1820 bytes) 2024-09-17T21:46:42.558 INFO:teuthology.orchestra.run.smithi165.stdout:1519 write 0x1a4ca00 thru 0x1a4cb6b (0x16c bytes) 2024-09-17T21:46:42.558 INFO:teuthology.orchestra.run.smithi165.stdout:1520 read 0x5c3400 thru 0x5cbc68 (0x8869 bytes) 2024-09-17T21:46:42.558 INFO:teuthology.orchestra.run.smithi165.stdout:1521 read 0x2e9b800 thru 0x2ea2937 (0x7138 bytes) 2024-09-17T21:46:42.577 INFO:teuthology.orchestra.run.smithi165.stdout:1523 write 0xb7ee000 thru 0xb7f4095 (0x6096 bytes) 2024-09-17T21:46:42.583 INFO:teuthology.orchestra.run.smithi165.stdout:1524 punch from 0xb151200 to 0xb157400, (0x6200 bytes) 2024-09-17T21:46:42.584 INFO:teuthology.orchestra.run.smithi165.stdout:1525 read 0x1aa9c00 thru 0x1ab986a (0xfc6b bytes) 2024-09-17T21:46:42.584 INFO:teuthology.orchestra.run.smithi165.stdout:1528 read 0x36fa800 thru 0x370156f (0x6d70 bytes) 2024-09-17T21:46:42.585 INFO:teuthology.orchestra.run.smithi165.stdout:1529 write 0x5c78400 thru 0x5c7bd9d (0x399e bytes) 2024-09-17T21:46:42.585 INFO:teuthology.orchestra.run.smithi165.stdout:1531 write 0x5548c00 thru 0x554a661 (0x1a62 bytes) 2024-09-17T21:46:42.585 INFO:teuthology.orchestra.run.smithi165.stdout:1532 read 0x7988200 thru 0x798e31a (0x611b bytes) 2024-09-17T21:46:42.585 INFO:teuthology.orchestra.run.smithi165.stdout:1535 read 0x16bba00 thru 0x16c14a3 (0x5aa4 bytes) 2024-09-17T21:46:42.586 INFO:teuthology.orchestra.run.smithi165.stdout:1538 punch from 0x6efb400 to 0x6f02400, (0x7000 bytes) 2024-09-17T21:46:42.586 INFO:teuthology.orchestra.run.smithi165.stdout:1539 write 0x2114600 thru 0x2120755 (0xc156 bytes) 2024-09-17T21:46:42.586 INFO:teuthology.orchestra.run.smithi165.stdout:1541 write 0x265200 thru 0x2705f5 (0xb3f6 bytes) 2024-09-17T21:46:42.587 INFO:teuthology.orchestra.run.smithi165.stdout:1542 read 0x8134e00 thru 0x813fce9 (0xaeea bytes) 2024-09-17T21:46:42.587 INFO:teuthology.orchestra.run.smithi165.stdout:1543 read 0x9951000 thru 0x9960578 (0xf579 bytes) 2024-09-17T21:46:42.587 INFO:teuthology.orchestra.run.smithi165.stdout:skipping zero length punch hole 2024-09-17T21:46:42.587 INFO:teuthology.orchestra.run.smithi165.stdout:1546 write 0x981200 thru 0x98fc29 (0xea2a bytes) 2024-09-17T21:46:42.588 INFO:teuthology.orchestra.run.smithi165.stdout:1547 punch from 0x5058e00 to 0x505ac00, (0x1e00 bytes) 2024-09-17T21:46:42.588 INFO:teuthology.orchestra.run.smithi165.stdout:1548 read 0x5006a00 thru 0x501042f (0x9a30 bytes) 2024-09-17T21:46:42.601 INFO:teuthology.orchestra.run.smithi165.stdout:1552 write 0xdec7c00 thru 0xdecb079 (0x347a bytes) 2024-09-17T21:46:42.601 INFO:teuthology.orchestra.run.smithi165.stdout:1553 write 0x5f29000 thru 0x5f31796 (0x8797 bytes) 2024-09-17T21:46:42.602 INFO:teuthology.orchestra.run.smithi165.stdout:1554 trunc from 0xdecb200 to 0xe517800 2024-09-17T21:46:42.614 INFO:teuthology.orchestra.run.smithi165.stdout:1555 write 0x1f33a00 thru 0x1f35c88 (0x2289 bytes) 2024-09-17T21:46:42.615 INFO:teuthology.orchestra.run.smithi165.stdout:1559 read 0x6712e00 thru 0x6715f40 (0x3141 bytes) 2024-09-17T21:46:42.617 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:42.615+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:42.623 INFO:teuthology.orchestra.run.smithi165.stdout:1562 trunc from 0xe517800 to 0x7587600 2024-09-17T21:46:42.647 INFO:teuthology.orchestra.run.smithi165.stdout:1563 trunc from 0x7587600 to 0xbed200 2024-09-17T21:46:42.652 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.652 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:42.664 INFO:teuthology.orchestra.run.smithi165.stdout:1566 punch from 0x517800 to 0x51a200, (0x2a00 bytes) 2024-09-17T21:46:42.665 INFO:teuthology.orchestra.run.smithi165.stdout:1567 read 0x325a00 thru 0x334374 (0xe975 bytes) 2024-09-17T21:46:42.671 INFO:teuthology.orchestra.run.smithi165.stdout:1572 trunc from 0xbed200 to 0x5739c00 2024-09-17T21:46:42.681 INFO:teuthology.orchestra.run.smithi165.stdout:1578 read 0x482cc00 thru 0x482dc0b (0x100c bytes) 2024-09-17T21:46:42.681 INFO:teuthology.orchestra.run.smithi165.stdout:1579 read 0x20b4400 thru 0x20beec9 (0xaaca bytes) 2024-09-17T21:46:42.681 INFO:teuthology.orchestra.run.smithi165.stdout:1580 read 0x2dd5600 thru 0x2dd66ad (0x10ae bytes) 2024-09-17T21:46:42.682 INFO:teuthology.orchestra.run.smithi165.stdout:1581 trunc from 0x5739c00 to 0x6db4800 2024-09-17T21:46:42.692 INFO:teuthology.orchestra.run.smithi165.stdout:1584 write 0x34f7800 thru 0x34faa20 (0x3221 bytes) 2024-09-17T21:46:42.701 INFO:teuthology.orchestra.run.smithi165.stdout:1586 trunc from 0x6db4800 to 0xe376000 2024-09-17T21:46:42.713 INFO:teuthology.orchestra.run.smithi165.stdout:1589 write 0x69e6a00 thru 0x69e865e (0x1c5f bytes) 2024-09-17T21:46:42.714 INFO:teuthology.orchestra.run.smithi165.stdout:1590 read 0xaf85000 thru 0xaf8cc59 (0x7c5a bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1591 write 0xe23ac00 thru 0xe23fab2 (0x4eb3 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1592 punch from 0xcab0600 to 0xcaba400, (0x9e00 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1594 read 0x4d08000 thru 0x4d119f1 (0x99f2 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1598 punch from 0xc468600 to 0xc473600, (0xb000 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1600 read 0x3380200 thru 0x338a5f3 (0xa3f4 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1603 punch from 0x528ac00 to 0x528f600, (0x4a00 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1606 read 0x6bc0e00 thru 0x6bc716a (0x636b bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1609 write 0xc1cf800 thru 0xc1db420 (0xbc21 bytes) 2024-09-17T21:46:42.715 INFO:teuthology.orchestra.run.smithi165.stdout:1610 read 0x88a2e00 thru 0x88b214a (0xf34b bytes) 2024-09-17T21:46:42.717 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:42.715+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:42.721 INFO:teuthology.orchestra.run.smithi165.stdout:1611 trunc from 0xe376000 to 0x8eb8200 2024-09-17T21:46:42.737 INFO:teuthology.orchestra.run.smithi165.stdout:1612 read 0x1095000 thru 0x109d0c9 (0x80ca bytes) 2024-09-17T21:46:42.739 INFO:teuthology.orchestra.run.smithi165.stdout:1616 read 0x35ff600 thru 0x3600302 (0xd03 bytes) 2024-09-17T21:46:42.739 INFO:teuthology.orchestra.run.smithi165.stdout:1618 read 0x8252600 thru 0x825cafb (0xa4fc bytes) 2024-09-17T21:46:42.739 INFO:teuthology.orchestra.run.smithi165.stdout:1620 read 0x65cc00 thru 0x668a86 (0xbe87 bytes) 2024-09-17T21:46:42.740 INFO:teuthology.orchestra.run.smithi165.stdout:1622 read 0x5ecf600 thru 0x5ed2095 (0x2a96 bytes) 2024-09-17T21:46:42.752 INFO:teuthology.orchestra.run.smithi165.stdout:1624 write 0xb5b5600 thru 0xb5c1663 (0xc064 bytes) 2024-09-17T21:46:42.753 INFO:teuthology.orchestra.run.smithi165.stdout:1625 read 0x5fee600 thru 0x5ff5b2d (0x752e bytes) 2024-09-17T21:46:42.753 INFO:teuthology.orchestra.run.smithi165.stdout:1627 write 0xb483000 thru 0xb48b50c (0x850d bytes) 2024-09-17T21:46:42.765 INFO:teuthology.orchestra.run.smithi165.stdout:1628 trunc from 0xb5c1800 to 0x1ad4c00 2024-09-17T21:46:42.765 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:46:42.781 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.781 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.792 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:42.795 INFO:teuthology.orchestra.run.smithi165.stdout:1630 write 0x1d6c400 thru 0x1d7aee9 (0xeaea bytes) 2024-09-17T21:46:42.796 INFO:teuthology.orchestra.run.smithi165.stdout:1631 read 0x9b200 thru 0xa25ea (0x73eb bytes) 2024-09-17T21:46:42.796 INFO:teuthology.orchestra.run.smithi165.stdout:1632 punch from 0xbc5000 to 0xbc7c00, (0x2c00 bytes) 2024-09-17T21:46:42.811 INFO:teuthology.orchestra.run.smithi165.stdout:1634 write 0x6a3d200 thru 0x6a3f570 (0x2371 bytes) 2024-09-17T21:46:42.811 INFO:teuthology.orchestra.run.smithi165.stdout:1636 punch from 0x415b600 to 0x4161000, (0x5a00 bytes) 2024-09-17T21:46:42.825 INFO:teuthology.orchestra.run.smithi165.stdout:1637 write 0xa5ff800 thru 0xa609dd2 (0xa5d3 bytes) 2024-09-17T21:46:42.826 INFO:teuthology.orchestra.run.smithi165.stdout:1638 write 0x4be5600 thru 0x4bf4b1d (0xf51e bytes) 2024-09-17T21:46:42.826 INFO:teuthology.orchestra.run.smithi165.stdout:1641 punch from 0xa5b3c00 to 0xa5bbe00, (0x8200 bytes) 2024-09-17T21:46:42.833 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:42.831+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:42.842 INFO:teuthology.orchestra.run.smithi165.stdout:1643 trunc from 0xa609e00 to 0x9af200 2024-09-17T21:46:42.862 INFO:teuthology.orchestra.run.smithi165.stdout:1645 read 0x95e200 thru 0x96d26c (0xf06d bytes) 2024-09-17T21:46:42.877 INFO:teuthology.orchestra.run.smithi165.stdout:1646 write 0x5e2aa00 thru 0x5e375ef (0xcbf0 bytes) 2024-09-17T21:46:42.898 INFO:teuthology.orchestra.run.smithi165.stdout:1650 write 0xc705a00 thru 0xc7128be (0xcebf bytes) 2024-09-17T21:46:42.899 INFO:teuthology.orchestra.run.smithi165.stdout:1651 punch from 0xd9d800 to 0xdab600, (0xde00 bytes) 2024-09-17T21:46:42.903 INFO:teuthology.orchestra.run.smithi165.stdout:1652 write 0xae72400 thru 0xae7a97e (0x857f bytes) 2024-09-17T21:46:42.903 INFO:teuthology.orchestra.run.smithi165.stdout:1659 write 0xa4a6600 thru 0xa4b173d (0xb13e bytes) 2024-09-17T21:46:42.903 INFO:teuthology.orchestra.run.smithi165.stdout:1660 write 0xbcff200 thru 0xbd0beff (0xcd00 bytes) 2024-09-17T21:46:42.916 INFO:teuthology.orchestra.run.smithi165.stdout:1661 write 0xd11a200 thru 0xd129bf1 (0xf9f2 bytes) 2024-09-17T21:46:42.918 INFO:teuthology.orchestra.run.smithi165.stdout:1663 read 0x9b96c00 thru 0x9b9ec62 (0x8063 bytes) 2024-09-17T21:46:42.920 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:42.917+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:42.922 INFO:teuthology.orchestra.run.smithi165.stdout:1667 trunc from 0xd129c00 to 0x9445a00 2024-09-17T21:46:42.944 INFO:teuthology.orchestra.run.smithi165.stdout:1670 write 0x606600 thru 0x60e9db (0x83dc bytes) 2024-09-17T21:46:42.945 INFO:teuthology.orchestra.run.smithi165.stdout:1674 read 0x6106000 thru 0x610a183 (0x4184 bytes) 2024-09-17T21:46:42.945 INFO:teuthology.orchestra.run.smithi165.stdout:1676 punch from 0x6134600 to 0x613a000, (0x5a00 bytes) 2024-09-17T21:46:42.945 INFO:teuthology.orchestra.run.smithi165.stdout:1679 clone 15 order 19 su 65536 sc 5 2024-09-17T21:46:42.948 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:42.948 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:42.958 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:43.021 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:43.019+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:43.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:43.065 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:43.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:43.121 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:43.119+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:43.145 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:43.145 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:43.162 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:43.179 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:43.179 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:43.184 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:43.184 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:43.184 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:43.196 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:43.213 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:43.222 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:43.219+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:43.230 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:43.301 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.301 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:43.301 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:43.301 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:43.301 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.310 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:43.323 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:43.321+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510767512000001, 2024-09-17T21:46:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510757211000001, 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510741137, 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510729924, 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510712485999999, 2024-09-17T21:46:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510701615999999, 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:43.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.510506739, 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:43.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:43.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.508802069000001, 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:43.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.508791430999999, 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:43.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.508781337999999, 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.508771219, 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:43.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:43.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:43.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.506640989000001, 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:43.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.506631257999999, 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:43.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:43.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.505847898999999, 2024-09-17T21:46:43.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:43.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:43.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.505832329, 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:43.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:43.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.505739427000002, 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:43.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:43.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.505244808, 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:43.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.505100763000002, 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:43.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.504415326, 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.504052936000001, 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:43.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.513 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:43.515 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:43.512+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:43.615 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:43.613+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:43.622 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.622 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:43.622 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:43.622 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:43.716 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:43.714+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:43.737 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:43.737 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:43.737 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:43.737 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:43.817 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:43.815+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:43.845 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone14 from 0x9445a00 (overlap 0x9af200) to 0x7bf400 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:43.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.054495928000001, 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:43.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:43.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:43.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:43.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.044844963999999, 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:46:43.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:910 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.044119753, 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082095099999999997, 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 910 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820301+0000", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820306+0000", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7180000000000004e-06 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820601+0000", 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029544599999999999 2024-09-17T21:46:43.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820609+0000", 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821192+0000", 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058298099999999999 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821301+0000", 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109826 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.828510+0000", 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072084130000000003 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2005 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.032228335999999, 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076004749999999998, 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2005 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:43.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832195+0000", 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.339e-06 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832201+0000", 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832206+0000", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3090000000000003e-06 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839559+0000", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073530169999999999 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839641+0000", 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1210000000000003e-05 2024-09-17T21:46:43.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839792+0000", 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015160099999999999 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2006 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:43.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.03215237, 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083420109999999999, 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2006 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:43.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832272+0000", 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832281+0000", 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2909999999999996e-06 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832288+0000", 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840364+0000", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080765279999999995 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840441+0000", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7028999999999997e-05 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840610+0000", 2024-09-17T21:46:43.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016856299999999999 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2033 3.8 3:109bf8c5:::rbd_data.11121f8e87bb.000000000000001c:8 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.031976615, 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077497670000000003, 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2033 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:43.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832444+0000", 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832446+0000", 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832453+0000", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832457+0000", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840020+0000", 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075630999999999997 2024-09-17T21:46:43.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840101+0000", 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0913000000000003e-05 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840193+0000", 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1823000000000005e-05 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.960899095, 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:46:43.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:46:43.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.95943696, 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:46:43.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:46:43.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.956825842000001, 2024-09-17T21:46:43.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:46:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:46:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4125 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.956097856, 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098331159999999994, 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4125 2024-09-17T21:46:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908323+0000", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908327+0000", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908334+0000", 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4880000000000001e-06 2024-09-17T21:46:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908342+0000", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8909999999999995e-06 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914269+0000", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059269379999999997 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914325+0000", 2024-09-17T21:46:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6508000000000002e-05 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914456+0000", 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130889 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918056+0000", 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035999629999999999 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918075+0000", 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8549000000000001e-05 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918119+0000", 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4535e-05 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918155+0000", 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5954000000000002e-05 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.892711385, 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:46:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:46:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.891983163000001, 2024-09-17T21:46:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:46:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:46:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.5584060270000002, 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:46:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9916475249999994, 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:46:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:46:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:46:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.9841253830000003, 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:46:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:350 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 5242880~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0172311250000003, 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077978400000000003, 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 350 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847191+0000", 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0679999999999999e-06 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847203+0000", 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1554e-05 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847206+0000", 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-06 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853739+0000", 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065326860000000002 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853817+0000", 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8072999999999996e-05 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.854987+0000", 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011696720000000001 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:351 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 16777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0172054769999992, 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080135269999999995, 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 351 2024-09-17T21:46:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847217+0000", 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.668e-06 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847225+0000", 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847229+0000", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5700000000000001e-06 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855031+0000", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078027970000000002 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855115+0000", 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3493999999999997e-05 2024-09-17T21:46:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855228+0000", 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001133 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:358 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 6291456~819200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0171555780000006, 2024-09-17T21:46:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095067050000000007, 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 358 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847267+0000", 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.238e-06 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847272+0000", 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0509999999999996e-06 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847275+0000", 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1209999999999998e-06 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855269+0000", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079939260000000002 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855350+0000", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0716000000000004e-05 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856771+0000", 2024-09-17T21:46:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014212719999999999 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:359 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.0171255499999994, 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096101729999999996, 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847297+0000", 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847302+0000", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847305+0000", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.072e-06 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856799+0000", 2024-09-17T21:46:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0094936829999999993 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856847+0000", 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7970000000000003e-05 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856905+0000", 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7819000000000003e-05 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4822.0:730 3.d e51/21 3:b2db4994:::rbd_data.111223014882.0000000000000000:head v 51'272, mlcod=51'270)", 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.828901933, 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096314799999999996, 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035519+0000", 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.44e-07 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039382+0000", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038629329999999998 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039396+0000", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4079e-05 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039402+0000", 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:46:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039448+0000", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100999999999998e-05 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039463+0000", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5537e-05 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045104+0000", 2024-09-17T21:46:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005640305 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045150+0000", 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6103000000000002e-05 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:43.976 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:43.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:43.978 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:43.975+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:44.079 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:44.077+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:44.092 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:44.092 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:44.093 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:44.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:44.180 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:44.177+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:44.213 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.213 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:44.213 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:44.213 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:44.213 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:44.222 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:44.226 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-17T21:46:44.305 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:44.303+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:44.377 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 57.215284318999998, 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:44.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:44.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.554299029999999, 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:46:44.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:46:44.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.536727639, 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:46:44.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.536693946, 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:46:44.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:46:44.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533852642999999, 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:46:44.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:46:44.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533775331999999, 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:46:44.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:46:44.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533752938999999, 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:46:44.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533731818, 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:46:44.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:46:44.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533711073999999, 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.53329643, 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:46:44.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:44.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.533261239, 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:46:44.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.532369775999999, 2024-09-17T21:46:44.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:46:44.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.532003706999999, 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:46:44.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:44.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.478696489000001, 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:46:44.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.478660717, 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:46:44.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.474522345, 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:46:44.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:46:44.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.469328623999999, 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:46:44.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.076822031000001, 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:46:44.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:46:44.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:44.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.5055222980000007, 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:44.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:46:44.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.4999749350000009, 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:46:44.474 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:46:44.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:46:44.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:46:44.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.480 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:44.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:44.483 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:44.479+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1681 read 0x17b6a00 thru 0x17c60fe (0xf6ff bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1682 write 0xa443a00 thru 0xa44f711 (0xbd12 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1685 write 0x7862800 thru 0x786404a (0x184b bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1687 read 0x2c4a000 thru 0x2c4a046 (0x47 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1688 punch from 0x2711800 to 0x2717400, (0x5c00 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1689 punch from 0x97b4200 to 0x97bf400, (0xb200 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1695 read 0x5ab1800 thru 0x5ab8fc4 (0x77c5 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1696 read 0x5560600 thru 0x556c2cd (0xbcce bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1697 punch from 0x4e7ae00 to 0x4e84400, (0x9600 bytes) 2024-09-17T21:46:44.484 INFO:teuthology.orchestra.run.smithi165.stdout:1700 write 0xcc89c00 thru 0xcc8e561 (0x4962 bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1701 write 0xf6f600 thru 0xf73134 (0x3b35 bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1702 read 0x2a37c00 thru 0x2a3f2fb (0x76fc bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1703 write 0x31a8a00 thru 0x31ab960 (0x2f61 bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1704 read 0xb333e00 thru 0xb3382fb (0x44fc bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1706 write 0x2858200 thru 0x285bdf9 (0x3bfa bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1712 write 0x5521a00 thru 0x5530ce7 (0xf2e8 bytes) 2024-09-17T21:46:44.485 INFO:teuthology.orchestra.run.smithi165.stdout:1714 trunc from 0xcc8e600 to 0x8825000 2024-09-17T21:46:44.522 INFO:teuthology.orchestra.run.smithi165.stdout:1715 write 0xe428c00 thru 0xe42e971 (0x5d72 bytes) 2024-09-17T21:46:44.529 INFO:teuthology.orchestra.run.smithi165.stdout:1716 punch from 0x970d200 to 0x9715000, (0x7e00 bytes) 2024-09-17T21:46:44.531 INFO:teuthology.orchestra.run.smithi165.stdout:1717 read 0x2b94e00 thru 0x2ba0f3e (0xc13f bytes) 2024-09-17T21:46:44.531 INFO:teuthology.orchestra.run.smithi165.stdout:1718 write 0x4ff2c00 thru 0x4ffdc02 (0xb003 bytes) 2024-09-17T21:46:44.531 INFO:teuthology.orchestra.run.smithi165.stdout:1719 read 0xb98de00 thru 0xb99096b (0x2b6c bytes) 2024-09-17T21:46:44.531 INFO:teuthology.orchestra.run.smithi165.stdout:1720 punch from 0x10f2000 to 0x10f6200, (0x4200 bytes) 2024-09-17T21:46:44.532 INFO:teuthology.orchestra.run.smithi165.stdout:1723 punch from 0x96e2600 to 0x96ec000, (0x9a00 bytes) 2024-09-17T21:46:44.532 INFO:teuthology.orchestra.run.smithi165.stdout:1728 write 0x375400 thru 0x37f4a8 (0xa0a9 bytes) 2024-09-17T21:46:44.532 INFO:teuthology.orchestra.run.smithi165.stdout:1729 punch from 0x7857a00 to 0x7864400, (0xca00 bytes) 2024-09-17T21:46:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:1732 read 0xeb9400 thru 0xebb308 (0x1f09 bytes) 2024-09-17T21:46:44.536 INFO:teuthology.orchestra.run.smithi165.stdout:1734 punch from 0x2361200 to 0x236ba00, (0xa800 bytes) 2024-09-17T21:46:44.536 INFO:teuthology.orchestra.run.smithi165.stdout:1738 read 0x524e600 thru 0x524fc80 (0x1681 bytes) 2024-09-17T21:46:44.536 INFO:teuthology.orchestra.run.smithi165.stdout:1740 read 0xcf72800 thru 0xcf7de0e (0xb60f bytes) 2024-09-17T21:46:44.537 INFO:teuthology.orchestra.run.smithi165.stdout:1741 write 0x5276c00 thru 0x52856be (0xeabf bytes) 2024-09-17T21:46:44.537 INFO:teuthology.orchestra.run.smithi165.stdout:1743 trunc from 0xe42ea00 to 0xe407e00 2024-09-17T21:46:44.555 INFO:teuthology.orchestra.run.smithi165.stdout:1744 write 0xc82cc00 thru 0xc834797 (0x7b98 bytes) 2024-09-17T21:46:44.562 INFO:teuthology.orchestra.run.smithi165.stdout:1745 read 0x220f400 thru 0x22119c0 (0x25c1 bytes) 2024-09-17T21:46:44.562 INFO:teuthology.orchestra.run.smithi165.stdout:1746 read 0xb927400 thru 0xb9327e1 (0xb3e2 bytes) 2024-09-17T21:46:44.576 INFO:teuthology.orchestra.run.smithi165.stdout:1747 trunc from 0xe407e00 to 0x1e7c800 2024-09-17T21:46:44.582 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:44.579+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:44.634 INFO:teuthology.orchestra.run.smithi165.stdout:1748 write 0xbf6ac00 thru 0xbf76ca1 (0xc0a2 bytes) 2024-09-17T21:46:44.639 INFO:teuthology.orchestra.run.smithi165.stdout:1753 read 0x59fe00 thru 0x5ab55a (0xb75b bytes) 2024-09-17T21:46:44.640 INFO:teuthology.orchestra.run.smithi165.stdout:1755 write 0xbe9aa00 thru 0xbea1f0d (0x750e bytes) 2024-09-17T21:46:44.657 INFO:teuthology.orchestra.run.smithi165.stdout:1756 write 0xdeb8200 thru 0xdec3753 (0xb554 bytes) 2024-09-17T21:46:44.658 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4974.0:743 3.1a 3.ac0e573a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:44.656562+0000", 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.0689999999999999e-05, 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0988e-05, 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4974", 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1788764723", 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 743 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656562+0000", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656562+0000", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656563+0000", 2024-09-17T21:46:44.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9500000000000001e-07 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656567+0000", 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.622e-06 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656580+0000", 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2911999999999999e-05 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:44.656587+0000", 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1609999999999997e-06 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:46:44.662 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:44.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:44.677 INFO:teuthology.orchestra.run.smithi165.stdout:1757 write 0xdf61000 thru 0xdf7026a (0xf26b bytes) 2024-09-17T21:46:44.677 INFO:teuthology.orchestra.run.smithi165.stdout:1758 read 0x54d2600 thru 0x54e2072 (0xfa73 bytes) 2024-09-17T21:46:44.678 INFO:teuthology.orchestra.run.smithi165.stdout:1760 punch from 0x7e1da00 to 0x7e21a00, (0x4000 bytes) 2024-09-17T21:46:44.678 INFO:teuthology.orchestra.run.smithi165.stdout:1761 write 0x9450c00 thru 0x9460a26 (0xfe27 bytes) 2024-09-17T21:46:44.678 INFO:teuthology.orchestra.run.smithi165.stdout:1764 read 0xcdb7200 thru 0xcdc2842 (0xb643 bytes) 2024-09-17T21:46:44.678 INFO:teuthology.orchestra.run.smithi165.stdout:1766 read 0x33f2a00 thru 0x33fd304 (0xa905 bytes) 2024-09-17T21:46:44.679 INFO:teuthology.orchestra.run.smithi165.stdout:1768 write 0x3ff4c00 thru 0x3ff606e (0x146f bytes) 2024-09-17T21:46:44.680 INFO:teuthology.orchestra.run.smithi165.stdout:1771 trunc from 0xdf70400 to 0xca7ee00 2024-09-17T21:46:44.683 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:44.680+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:44.712 INFO:teuthology.orchestra.run.smithi165.stdout:1773 trunc from 0xca7ee00 to 0x6b55000 2024-09-17T21:46:44.735 INFO:teuthology.orchestra.run.smithi165.stdout:1775 read 0x3561e00 thru 0x356f805 (0xda06 bytes) 2024-09-17T21:46:44.735 INFO:teuthology.orchestra.run.smithi165.stdout:1776 read 0x450ec00 thru 0x45168f3 (0x7cf4 bytes) 2024-09-17T21:46:44.736 INFO:teuthology.orchestra.run.smithi165.stdout:1777 read 0x1d6ce00 thru 0x1d75f83 (0x9184 bytes) 2024-09-17T21:46:44.740 INFO:teuthology.orchestra.run.smithi165.stdout:1778 trunc from 0x6b55000 to 0xacd4000 2024-09-17T21:46:44.763 INFO:teuthology.orchestra.run.smithi165.stdout:1779 trunc from 0xacd4000 to 0x3ad2a00 2024-09-17T21:46:44.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:46:44.784 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:44.781+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:44.787 INFO:teuthology.orchestra.run.smithi165.stdout:1781 trunc from 0x3ad2a00 to 0xbeab600 2024-09-17T21:46:44.796 INFO:teuthology.orchestra.run.smithi165.stdout:1782 punch from 0x1e7a800 to 0x1e86200, (0xba00 bytes) 2024-09-17T21:46:44.801 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.801 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:44.801 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:44.801 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:44.801 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:44.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:44.811 INFO:teuthology.orchestra.run.smithi165.stdout:1783 trunc from 0xbeab600 to 0x1419400 2024-09-17T21:46:44.833 INFO:teuthology.orchestra.run.smithi165.stdout:1784 trunc from 0x1419400 to 0x2f12600 2024-09-17T21:46:44.849 INFO:teuthology.orchestra.run.smithi165.stdout:1785 trunc from 0x2f12600 to 0x948000 2024-09-17T21:46:44.876 INFO:teuthology.orchestra.run.smithi165.stdout:1789 read 0x82ae00 thru 0x834982 (0x9b83 bytes) 2024-09-17T21:46:44.885 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:44.882+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:44.895 INFO:teuthology.orchestra.run.smithi165.stdout:1790 trunc from 0x948000 to 0xc1b4a00 2024-09-17T21:46:44.903 INFO:teuthology.orchestra.run.smithi165.stdout:1791 write 0x2587c00 thru 0x2596eed (0xf2ee bytes) 2024-09-17T21:46:44.905 INFO:teuthology.orchestra.run.smithi165.stdout:1793 write 0x8493c00 thru 0x84a027e (0xc67f bytes) 2024-09-17T21:46:44.905 INFO:teuthology.orchestra.run.smithi165.stdout:1795 write 0x9c9be00 thru 0x9ca5c75 (0x9e76 bytes) 2024-09-17T21:46:44.905 INFO:teuthology.orchestra.run.smithi165.stdout:1796 punch from 0x4ce8a00 to 0x4cf5400, (0xca00 bytes) 2024-09-17T21:46:44.906 INFO:teuthology.orchestra.run.smithi165.stdout:1797 punch from 0x6360200 to 0x6366600, (0x6400 bytes) 2024-09-17T21:46:44.908 INFO:teuthology.orchestra.run.smithi165.stdout:1799 trunc from 0xc1b4a00 to 0xcf80400 2024-09-17T21:46:44.928 INFO:teuthology.orchestra.run.smithi165.stdout:1800 punch from 0xabe0000 to 0xabe4800, (0x4800 bytes) 2024-09-17T21:46:44.937 INFO:teuthology.orchestra.run.smithi165.stdout:1801 trunc from 0xcf80400 to 0x8c82e00 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.991999315000001, 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:44.947 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:44.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:44.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.989974263000001, 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:44.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:44.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.956 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.975988207, 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:44.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.974027, 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.971993643000001, 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.970015725, 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.974 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.969992931, 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.964083867999999, 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.979 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:44.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:44.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:44.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.962052808999999, 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:44.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.962026235, 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:44.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.993 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.961416442000001, 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:44.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:44.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:44.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:44.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.961398855999999, 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:44.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:45.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:45.004 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.961383899000001, 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:45.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:45.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.010 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.245702411, 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:45.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:45.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.245671789999999, 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:45.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:45.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.245647172000002, 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:45.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:45.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:45.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.245625225000001, 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:45.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:45.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.245603755000001, 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:45.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:45.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.204186952000001, 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:46:45.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:46:45.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.203142492000001, 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:45.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:45.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:45.050 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:45.051 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:45.048+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:45.052 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1802 read 0x1b99600 thru 0x1ba04e2 (0x6ee3 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1803 punch from 0x79bfa00 to 0x79cb400, (0xba00 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1804 write 0x75f3600 thru 0x75fe845 (0xb246 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1807 read 0x752aa00 thru 0x752b934 (0xf35 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1808 write 0x5d9f400 thru 0x5da425d (0x4e5e bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1809 punch from 0x3d56200 to 0x3d5c800, (0x6600 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1810 read 0x3bb7000 thru 0x3bc27fd (0xb7fe bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1813 punch from 0x5214000 to 0x5221a00, (0xda00 bytes) 2024-09-17T21:46:45.054 INFO:teuthology.orchestra.run.smithi165.stdout:1814 write 0x71e6000 thru 0x71e6d62 (0xd63 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1815 trunc from 0x8c82e00 to 0xbffd000 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1818 write 0x44c6c00 thru 0x44c7a30 (0xe31 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1819 write 0x8afe00 thru 0x8b7239 (0x743a bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1820 trunc from 0xbffd000 to 0x9dfe000 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1821 trunc from 0x9dfe000 to 0x7071600 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1822 read 0x1e8c400 thru 0x1e92e01 (0x6a02 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1824 write 0x62ae800 thru 0x62b8785 (0x9f86 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1825 read 0x35d400 thru 0x36b30d (0xdf0e bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1826 write 0x544c000 thru 0x544d329 (0x132a bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1827 read 0x4275400 thru 0x4280f9d (0xbb9e bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1829 read 0x3ff1c00 thru 0x3ff23cf (0x7d0 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1833 punch from 0x3d7ac00 to 0x3d83a00, (0x8e00 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1834 punch from 0x2c9f400 to 0x2ca7a00, (0x8600 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1835 write 0xe114000 thru 0xe1189e1 (0x49e2 bytes) 2024-09-17T21:46:45.055 INFO:teuthology.orchestra.run.smithi165.stdout:1837 read 0xb2bb200 thru 0xb2c436b (0x916c bytes) 2024-09-17T21:46:45.056 INFO:teuthology.orchestra.run.smithi165.stdout:1840 trunc from 0xe118a00 to 0x77b1a00 2024-09-17T21:46:45.064 INFO:teuthology.orchestra.run.smithi165.stdout:1844 read 0x1eb5c00 thru 0x1eba8e9 (0x4cea bytes) 2024-09-17T21:46:45.076 INFO:teuthology.orchestra.run.smithi165.stdout:1845 trunc from 0x77b1a00 to 0x2e52800 2024-09-17T21:46:45.096 INFO:teuthology.orchestra.run.smithi165.stdout:1846 read 0x24e000 thru 0x24f551 (0x1552 bytes) 2024-09-17T21:46:45.119 INFO:teuthology.orchestra.run.smithi165.stdout:1847 write 0xd902400 thru 0xd91115a (0xed5b bytes) 2024-09-17T21:46:45.121 INFO:teuthology.orchestra.run.smithi165.stdout:1849 read 0x1bd2200 thru 0x1bdc373 (0xa174 bytes) 2024-09-17T21:46:45.121 INFO:teuthology.orchestra.run.smithi165.stdout:1854 punch from 0xb627c00 to 0xb632000, (0xa400 bytes) 2024-09-17T21:46:45.122 INFO:teuthology.orchestra.run.smithi165.stdout:1856 read 0x81ac200 thru 0x81b55f7 (0x93f8 bytes) 2024-09-17T21:46:45.122 INFO:teuthology.orchestra.run.smithi165.stdout:1860 read 0x5ac1400 thru 0x5ad03b4 (0xefb5 bytes) 2024-09-17T21:46:45.122 INFO:teuthology.orchestra.run.smithi165.stdout:1863 punch from 0xd732400 to 0xd738600, (0x6200 bytes) 2024-09-17T21:46:45.123 INFO:teuthology.orchestra.run.smithi165.stdout:1864 punch from 0x2b6400 to 0x2c0a00, (0xa600 bytes) 2024-09-17T21:46:45.124 INFO:teuthology.orchestra.run.smithi165.stdout:1865 trunc from 0xd911200 to 0xbfae600 2024-09-17T21:46:45.146 INFO:teuthology.orchestra.run.smithi165.stdout:1866 read 0x353600 thru 0x35486f (0x1270 bytes) 2024-09-17T21:46:45.151 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:45.149+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:45.153 INFO:teuthology.orchestra.run.smithi165.stdout:1867 read 0x2efe000 thru 0x2f04c8c (0x6c8d bytes) 2024-09-17T21:46:45.153 INFO:teuthology.orchestra.run.smithi165.stdout:1869 read 0x77c1800 thru 0x77c20a6 (0x8a7 bytes) 2024-09-17T21:46:45.154 INFO:teuthology.orchestra.run.smithi165.stdout:1877 write 0x6dc3200 thru 0x6dccc70 (0x9a71 bytes) 2024-09-17T21:46:45.154 INFO:teuthology.orchestra.run.smithi165.stdout:1878 write 0x4d53800 thru 0x4d576cf (0x3ed0 bytes) 2024-09-17T21:46:45.154 INFO:teuthology.orchestra.run.smithi165.stdout:1879 read 0x2f2e00 thru 0x2fe4b7 (0xb6b8 bytes) 2024-09-17T21:46:45.171 INFO:teuthology.orchestra.run.smithi165.stdout:1880 write 0xe418800 thru 0xe4232c4 (0xaac5 bytes) 2024-09-17T21:46:45.175 INFO:teuthology.orchestra.run.smithi165.stdout:1881 trunc from 0xe423400 to 0xb726000 2024-09-17T21:46:45.193 INFO:teuthology.orchestra.run.smithi165.stdout:1882 read 0x1407e00 thru 0x1416902 (0xeb03 bytes) 2024-09-17T21:46:45.195 INFO:teuthology.orchestra.run.smithi165.stdout:1883 punch from 0x7d14400 to 0x7d22800, (0xe400 bytes) 2024-09-17T21:46:45.196 INFO:teuthology.orchestra.run.smithi165.stdout:1884 read 0x31ed200 thru 0x31f4896 (0x7697 bytes) 2024-09-17T21:46:45.196 INFO:teuthology.orchestra.run.smithi165.stdout:1885 write 0x5824800 thru 0x5829df3 (0x55f4 bytes) 2024-09-17T21:46:45.201 INFO:teuthology.orchestra.run.smithi165.stdout:1888 trunc from 0xb726000 to 0x7a13e00 2024-09-17T21:46:45.218 INFO:teuthology.orchestra.run.smithi165.stdout:1892 read 0x3938000 thru 0x393ed04 (0x6d05 bytes) 2024-09-17T21:46:45.219 INFO:teuthology.orchestra.run.smithi165.stdout:1893 read 0x1bb3c00 thru 0x1bc001e (0xc41f bytes) 2024-09-17T21:46:45.219 INFO:teuthology.orchestra.run.smithi165.stdout:1894 read 0x35ca000 thru 0x35d0037 (0x6038 bytes) 2024-09-17T21:46:45.219 INFO:teuthology.orchestra.run.smithi165.stdout:1895 read 0x4a06400 thru 0x4a06673 (0x274 bytes) 2024-09-17T21:46:45.220 INFO:teuthology.orchestra.run.smithi165.stdout:1897 read 0x63ca200 thru 0x63d6aca (0xc8cb bytes) 2024-09-17T21:46:45.220 INFO:teuthology.orchestra.run.smithi165.stdout:1898 read 0x65ee600 thru 0x65f3b3d (0x553e bytes) 2024-09-17T21:46:45.220 INFO:teuthology.orchestra.run.smithi165.stdout:1901 punch from 0x77ac00 to 0x788200, (0xd600 bytes) 2024-09-17T21:46:45.220 INFO:teuthology.orchestra.run.smithi165.stdout:1902 flatten 2024-09-17T21:46:45.226 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:45.227 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:45.227 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:45.227 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:45.240 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:45.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:45.261 INFO:teuthology.orchestra.run.smithi165.stdout:1907 trunc from 0x7a13e00 to 0xbff7800 2024-09-17T21:46:45.272 INFO:teuthology.orchestra.run.smithi165.stdout:1908 punch from 0x2cb1a00 to 0x2cbea00, (0xd000 bytes) 2024-09-17T21:46:45.273 INFO:teuthology.orchestra.run.smithi165.stdout:1912 write 0x7c6a000 thru 0x7c776f6 (0xd6f7 bytes) 2024-09-17T21:46:45.273 INFO:teuthology.orchestra.run.smithi165.stdout:1914 punch from 0x138a000 to 0x1399000, (0xf000 bytes) 2024-09-17T21:46:45.274 INFO:teuthology.orchestra.run.smithi165.stdout:1915 punch from 0x9b64200 to 0x9b6ca00, (0x8800 bytes) 2024-09-17T21:46:45.274 INFO:teuthology.orchestra.run.smithi165.stdout:1917 write 0x6cb5000 thru 0x6cb60b8 (0x10b9 bytes) 2024-09-17T21:46:45.274 INFO:teuthology.orchestra.run.smithi165.stdout:1918 read 0x4948600 thru 0x494f47d (0x6e7e bytes) 2024-09-17T21:46:45.274 INFO:teuthology.orchestra.run.smithi165.stdout:1919 read 0x4072000 thru 0x407d3fb (0xb3fc bytes) 2024-09-17T21:46:45.275 INFO:teuthology.orchestra.run.smithi165.stdout:1920 punch from 0x6dd4800 to 0x6de3000, (0xe800 bytes) 2024-09-17T21:46:45.275 INFO:teuthology.orchestra.run.smithi165.stdout:1921 trunc from 0xbff7800 to 0xbee0400 2024-09-17T21:46:45.287 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:45.285+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:45.307 INFO:teuthology.orchestra.run.smithi165.stdout:1922 trunc from 0xbee0400 to 0x7b2a00 2024-09-17T21:46:45.342 INFO:teuthology.orchestra.run.smithi165.stdout:1923 trunc from 0x7b2a00 to 0x9af3200 2024-09-17T21:46:45.352 INFO:teuthology.orchestra.run.smithi165.stdout:1924 read 0x7f0ca00 thru 0x7f17e8b (0xb48c bytes) 2024-09-17T21:46:45.352 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:45.350+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1925 read 0x70fae00 thru 0x7100624 (0x5825 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1926 write 0x224fc00 thru 0x225a671 (0xaa72 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1928 write 0x3181600 thru 0x318ef5d (0xd95e bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1930 read 0x8111e00 thru 0x8114be7 (0x2de8 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1931 read 0x8dc5200 thru 0x8dc9e57 (0x4c58 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1933 write 0x5380000 thru 0x5387596 (0x7597 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1936 read 0x6343400 thru 0x6348819 (0x541a bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1937 read 0x5ef2a00 thru 0x5ef3c4f (0x1250 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1940 read 0x5815c00 thru 0x58182d7 (0x26d8 bytes) 2024-09-17T21:46:45.359 INFO:teuthology.orchestra.run.smithi165.stdout:1941 write 0x55b5200 thru 0x55bce22 (0x7c23 bytes) 2024-09-17T21:46:45.360 INFO:teuthology.orchestra.run.smithi165.stdout:1942 trunc from 0x9af3200 to 0x8e60e00 2024-09-17T21:46:45.369 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:45.370 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:45.370 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:45.370 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:45.377 INFO:teuthology.orchestra.run.smithi165.stdout:1944 punch from 0x7430600 to 0x743e800, (0xe200 bytes) 2024-09-17T21:46:45.379 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:45.380 INFO:teuthology.orchestra.run.smithi165.stdout:1945 read 0x5ab4600 thru 0x5abc1c7 (0x7bc8 bytes) 2024-09-17T21:46:45.380 INFO:teuthology.orchestra.run.smithi165.stdout:1946 punch from 0x1834600 to 0x183fc00, (0xb600 bytes) 2024-09-17T21:46:45.382 INFO:teuthology.orchestra.run.smithi165.stdout:1947 write 0x827a800 thru 0x827fc92 (0x5493 bytes) 2024-09-17T21:46:45.382 INFO:teuthology.orchestra.run.smithi165.stdout:1948 trunc from 0x8e60e00 to 0xc0ec200 2024-09-17T21:46:45.387 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:45.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:45.395 INFO:teuthology.orchestra.run.smithi165.stdout:1952 write 0xb2d2e00 thru 0xb2de34c (0xb54d bytes) 2024-09-17T21:46:45.402 INFO:teuthology.orchestra.run.smithi165.stdout:1953 punch from 0x7a71200 to 0x7a7f000, (0xde00 bytes) 2024-09-17T21:46:45.402 INFO:teuthology.orchestra.run.smithi165.stdout:1954 read 0x3cefc00 thru 0x3cf1a5c (0x1e5d bytes) 2024-09-17T21:46:45.402 INFO:teuthology.orchestra.run.smithi165.stdout:1958 clone 16 order 22 su 2097152 sc 5 2024-09-17T21:46:45.403 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:45.403 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:45.421 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:45.421 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:45.437 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:45.438 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:45.454 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:45.463 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:45.461+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:45.527 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:45.527 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:45.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:45.579 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:45.577+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:45.644 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:45.644 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:45.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:45.654 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:45.651+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:45.754 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:45.752+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:45.773 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:46:45.807 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:45.807 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:45.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:45.844 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:45.844 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:45.844 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 3, 4, 5] 2024-09-17T21:46:45.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-17T21:46:45.857 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:45.855+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:45.956 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:45.953+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:45.965 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:45.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:46.057 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:46.055+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:46.128 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.128 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:46.165 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:46.162+0000 7f0c28614640 -1 mon.a@0(leader).osd e58 definitely_dead 0 2024-09-17T21:46:46.165 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:46.163+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:46.207 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-17T21:46:46.247 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.247 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.255 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:46.296 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:46.294+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:46.301 INFO:teuthology.orchestra.run.smithi165.stdout:1960 read 0x6b67000 thru 0x6b72168 (0xb169 bytes) 2024-09-17T21:46:46.302 INFO:teuthology.orchestra.run.smithi165.stdout:1961 trunc from 0xc0ec200 to 0xbfc3c00 2024-09-17T21:46:46.320 INFO:teuthology.orchestra.run.smithi165.stdout:1962 read 0x8e4fe00 thru 0x8e5ee6f (0xf070 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1963 punch from 0x5c8a400 to 0x5c92600, (0x8200 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1967 punch from 0xaa5cc00 to 0xaa62e00, (0x6200 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1969 read 0x8b1ba00 thru 0x8b1c0c4 (0x6c5 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1970 punch from 0xb5cd400 to 0xb5d5400, (0x8000 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1971 write 0x98d0600 thru 0x98d7c6d (0x766e bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1972 read 0x8442800 thru 0x84468f3 (0x40f4 bytes) 2024-09-17T21:46:46.322 INFO:teuthology.orchestra.run.smithi165.stdout:1975 write 0xba06600 thru 0xba0f4df (0x8ee0 bytes) 2024-09-17T21:46:46.323 INFO:teuthology.orchestra.run.smithi165.stdout:1977 write 0x2a96a00 thru 0x2a9ab6d (0x416e bytes) 2024-09-17T21:46:46.323 INFO:teuthology.orchestra.run.smithi165.stdout:1979 write 0xad8e800 thru 0xad9bab2 (0xd2b3 bytes) 2024-09-17T21:46:46.323 INFO:teuthology.orchestra.run.smithi165.stdout:1980 read 0x6616200 thru 0x6618d35 (0x2b36 bytes) 2024-09-17T21:46:46.323 INFO:teuthology.orchestra.run.smithi165.stdout:1981 write 0x4f8be00 thru 0x4f92a1a (0x6c1b bytes) 2024-09-17T21:46:46.323 INFO:teuthology.orchestra.run.smithi165.stdout:1983 punch from 0x7823400 to 0x7824a00, (0x1600 bytes) 2024-09-17T21:46:46.341 INFO:teuthology.orchestra.run.smithi165.stdout:1985 write 0xdb9d800 thru 0xdba2230 (0x4a31 bytes) 2024-09-17T21:46:46.341 INFO:teuthology.orchestra.run.smithi165.stdout:1988 read 0xc820e00 thru 0xc826d59 (0x5f5a bytes) 2024-09-17T21:46:46.341 INFO:teuthology.orchestra.run.smithi165.stdout:1992 punch from 0x650be00 to 0x6514a00, (0x8c00 bytes) 2024-09-17T21:46:46.342 INFO:teuthology.orchestra.run.smithi165.stdout:1997 read 0x356f000 thru 0x3575e83 (0x6e84 bytes) 2024-09-17T21:46:46.342 INFO:teuthology.orchestra.run.smithi165.stdout:1998 write 0xd8c5400 thru 0xd8cb14b (0x5d4c bytes) 2024-09-17T21:46:46.349 INFO:teuthology.orchestra.run.smithi165.stdout:1999 trunc from 0xdba2400 to 0x79dbe00 2024-09-17T21:46:46.359 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:46.357+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:46.369 INFO:teuthology.orchestra.run.smithi165.stdout:2000 read 0x3d73e00 thru 0x3d747ce (0x9cf bytes) 2024-09-17T21:46:46.384 INFO:teuthology.orchestra.run.smithi165.stdout:2001 write 0xb554e00 thru 0xb564c70 (0xfe71 bytes) 2024-09-17T21:46:46.384 INFO:teuthology.orchestra.run.smithi165.stdout:2002 read 0x7a5d400 thru 0x7a6325d (0x5e5e bytes) 2024-09-17T21:46:46.384 INFO:teuthology.orchestra.run.smithi165.stdout:2005 write 0x1350e00 thru 0x135e1ab (0xd3ac bytes) 2024-09-17T21:46:46.385 INFO:teuthology.orchestra.run.smithi165.stdout:2010 read 0x3f22e00 thru 0x3f24078 (0x1279 bytes) 2024-09-17T21:46:46.386 INFO:teuthology.orchestra.run.smithi165.stdout:2011 trunc from 0xb564e00 to 0xcdba800 2024-09-17T21:46:46.411 INFO:teuthology.orchestra.run.smithi165.stdout:2014 write 0xd858400 thru 0xd865b9c (0xd79d bytes) 2024-09-17T21:46:46.412 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.412 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.417 INFO:teuthology.orchestra.run.smithi165.stdout:2015 read 0x5c27400 thru 0x5c2d438 (0x6039 bytes) 2024-09-17T21:46:46.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:46.427 INFO:teuthology.orchestra.run.smithi165.stdout:2016 trunc from 0xd865c00 to 0x582ac00 2024-09-17T21:46:46.454 INFO:teuthology.orchestra.run.smithi165.stdout:2017 trunc from 0x582ac00 to 0x2f09c00 2024-09-17T21:46:46.460 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:46.457+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:46.473 INFO:teuthology.orchestra.run.smithi165.stdout:2018 read 0x10d9600 thru 0x10e8127 (0xeb28 bytes) 2024-09-17T21:46:46.498 INFO:teuthology.orchestra.run.smithi165.stdout:2019 write 0xe7a1a00 thru 0xe7a46e5 (0x2ce6 bytes) 2024-09-17T21:46:46.499 INFO:teuthology.orchestra.run.smithi165.stdout:2020 write 0x4ef2200 thru 0x4ef602d (0x3e2e bytes) 2024-09-17T21:46:46.499 INFO:teuthology.orchestra.run.smithi165.stdout:2021 write 0x46ffc00 thru 0x470b7be (0xbbbf bytes) 2024-09-17T21:46:46.499 INFO:teuthology.orchestra.run.smithi165.stdout:2022 punch from 0x16d4c00 to 0x16e3600, (0xea00 bytes) 2024-09-17T21:46:46.499 INFO:teuthology.orchestra.run.smithi165.stdout:2024 read 0x2ac5c00 thru 0x2ad070e (0xab0f bytes) 2024-09-17T21:46:46.500 INFO:teuthology.orchestra.run.smithi165.stdout:2025 write 0xd858c00 thru 0xd85e2ea (0x56eb bytes) 2024-09-17T21:46:46.500 INFO:teuthology.orchestra.run.smithi165.stdout:2026 write 0x6eb2e00 thru 0x6ebe3c3 (0xb5c4 bytes) 2024-09-17T21:46:46.500 INFO:teuthology.orchestra.run.smithi165.stdout:2028 read 0xae1bc00 thru 0xae1fb6f (0x3f70 bytes) 2024-09-17T21:46:46.500 INFO:teuthology.orchestra.run.smithi165.stdout:2029 write 0x4d82200 thru 0x4d82f9b (0xd9c bytes) 2024-09-17T21:46:46.504 INFO:teuthology.orchestra.run.smithi165.stdout:2031 trunc from 0xe7a4800 to 0xab6ce00 2024-09-17T21:46:46.523 INFO:teuthology.orchestra.run.smithi165.stdout:2032 write 0x4bb4400 thru 0x4bc06a2 (0xc2a3 bytes) 2024-09-17T21:46:46.524 INFO:teuthology.orchestra.run.smithi165.stdout:2034 read 0x606fa00 thru 0x607316d (0x376e bytes) 2024-09-17T21:46:46.524 INFO:teuthology.orchestra.run.smithi165.stdout:2035 read 0x81a9600 thru 0x81b1d82 (0x8783 bytes) 2024-09-17T21:46:46.524 INFO:teuthology.orchestra.run.smithi165.stdout:2036 punch from 0x5d54e00 to 0x5d62c00, (0xde00 bytes) 2024-09-17T21:46:46.531 INFO:teuthology.orchestra.run.smithi165.stdout:2037 trunc from 0xab6ce00 to 0x4c54e00 2024-09-17T21:46:46.538 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.538 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:46.556 INFO:teuthology.orchestra.run.smithi165.stdout:2041 write 0x59d5800 thru 0x59e2b47 (0xd348 bytes) 2024-09-17T21:46:46.556 INFO:teuthology.orchestra.run.smithi165.stdout:2042 punch from 0x42eac00 to 0x42f0a00, (0x5e00 bytes) 2024-09-17T21:46:46.561 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:46.559+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:46.570 INFO:teuthology.orchestra.run.smithi165.stdout:2043 write 0x7d72200 thru 0x7d753fd (0x31fe bytes) 2024-09-17T21:46:46.570 INFO:teuthology.orchestra.run.smithi165.stdout:2044 punch from 0x7457c00 to 0x7461400, (0x9800 bytes) 2024-09-17T21:46:46.583 INFO:teuthology.orchestra.run.smithi165.stdout:2047 write 0x8678400 thru 0x8683312 (0xaf13 bytes) 2024-09-17T21:46:46.583 INFO:teuthology.orchestra.run.smithi165.stdout:2049 read 0x5894c00 thru 0x589efc4 (0xa3c5 bytes) 2024-09-17T21:46:46.584 INFO:teuthology.orchestra.run.smithi165.stdout:2051 read 0x47e0a00 thru 0x47eba6a (0xb06b bytes) 2024-09-17T21:46:46.588 INFO:teuthology.orchestra.run.smithi165.stdout:2052 trunc from 0x8683400 to 0xb464000 2024-09-17T21:46:46.598 INFO:teuthology.orchestra.run.smithi165.stdout:2054 write 0x9ab8000 thru 0x9abbdec (0x3ded bytes) 2024-09-17T21:46:46.602 INFO:teuthology.orchestra.run.smithi165.stdout:2055 trunc from 0xb464000 to 0x8a6a600 2024-09-17T21:46:46.617 INFO:teuthology.orchestra.run.smithi165.stdout:2056 read 0x1ee3000 thru 0x1eeab66 (0x7b67 bytes) 2024-09-17T21:46:46.618 INFO:teuthology.orchestra.run.smithi165.stdout:2058 punch from 0x2624800 to 0x262ba00, (0x7200 bytes) 2024-09-17T21:46:46.618 INFO:teuthology.orchestra.run.smithi165.stdout:2059 write 0x3ce1a00 thru 0x3cef574 (0xdb75 bytes) 2024-09-17T21:46:46.619 INFO:teuthology.orchestra.run.smithi165.stdout:2060 write 0x25d5400 thru 0x25d7f46 (0x2b47 bytes) 2024-09-17T21:46:46.619 INFO:teuthology.orchestra.run.smithi165.stdout:2061 punch from 0x3336600 to 0x3343200, (0xcc00 bytes) 2024-09-17T21:46:46.619 INFO:teuthology.orchestra.run.smithi165.stdout:2062 punch from 0x4c69800 to 0x4c6ce00, (0x3600 bytes) 2024-09-17T21:46:46.621 INFO:teuthology.orchestra.run.smithi165.stdout:2063 trunc from 0x8a6a600 to 0x6c63c00 2024-09-17T21:46:46.637 INFO:teuthology.orchestra.run.smithi165.stdout:2066 read 0x2dc4600 thru 0x2dcfd5d (0xb75e bytes) 2024-09-17T21:46:46.644 INFO:teuthology.orchestra.run.smithi165.stdout:2068 write 0x1203c00 thru 0x120db99 (0x9f9a bytes) 2024-09-17T21:46:46.645 INFO:teuthology.orchestra.run.smithi165.stdout:2069 write 0x152b200 thru 0x1536b4d (0xb94e bytes) 2024-09-17T21:46:46.645 INFO:teuthology.orchestra.run.smithi165.stdout:2071 read 0x380d600 thru 0x3816dae (0x97af bytes) 2024-09-17T21:46:46.645 INFO:teuthology.orchestra.run.smithi165.stdout:2072 read 0x1b45800 thru 0x1b497fb (0x3ffc bytes) 2024-09-17T21:46:46.646 INFO:teuthology.orchestra.run.smithi165.stdout:2074 punch from 0x64f1600 to 0x64f4a00, (0x3400 bytes) 2024-09-17T21:46:46.646 INFO:teuthology.orchestra.run.smithi165.stdout:2079 write 0x3f8bc00 thru 0x3f8ca25 (0xe26 bytes) 2024-09-17T21:46:46.649 INFO:teuthology.orchestra.run.smithi165.stdout:2080 trunc from 0x6c63c00 to 0x44d7e00 2024-09-17T21:46:46.663 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:46.660+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:46.663 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.663 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.666 INFO:teuthology.orchestra.run.smithi165.stdout:2081 write 0x2be7000 thru 0x2bea800 (0x3801 bytes) 2024-09-17T21:46:46.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:46.672 INFO:teuthology.orchestra.run.smithi165.stdout:2083 trunc from 0x44d7e00 to 0x906a400 2024-09-17T21:46:46.690 INFO:teuthology.orchestra.run.smithi165.stdout:2084 trunc from 0x906a400 to 0x2d42e00 2024-09-17T21:46:46.725 INFO:teuthology.orchestra.run.smithi165.stdout:2085 write 0xa3ebe00 thru 0xa3ed60e (0x180f bytes) 2024-09-17T21:46:46.726 INFO:teuthology.orchestra.run.smithi165.stdout:2086 write 0xe90200 thru 0xe92dd6 (0x2bd7 bytes) 2024-09-17T21:46:46.726 INFO:teuthology.orchestra.run.smithi165.stdout:2088 read 0x96c8e00 thru 0x96cacb4 (0x1eb5 bytes) 2024-09-17T21:46:46.726 INFO:teuthology.orchestra.run.smithi165.stdout:2090 write 0x6dc0800 thru 0x6dc9cc2 (0x94c3 bytes) 2024-09-17T21:46:46.750 INFO:teuthology.orchestra.run.smithi165.stdout:2093 write 0xcdb1800 thru 0xcdc1165 (0xf966 bytes) 2024-09-17T21:46:46.756 INFO:teuthology.orchestra.run.smithi165.stdout:2096 trunc from 0xcdc1200 to 0x8e12e00 2024-09-17T21:46:46.764 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:46.761+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:46.771 INFO:teuthology.orchestra.run.smithi165.stdout:2098 read 0x4b09a00 thru 0x4b151b3 (0xb7b4 bytes) 2024-09-17T21:46:46.772 INFO:teuthology.orchestra.run.smithi165.stdout:2099 read 0x3c00200 thru 0x3c04a4d (0x484e bytes) 2024-09-17T21:46:46.772 INFO:teuthology.orchestra.run.smithi165.stdout:2101 read 0x6234e00 thru 0x623b8b1 (0x6ab2 bytes) 2024-09-17T21:46:46.779 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.4. 2024-09-17T21:46:46.782 INFO:teuthology.orchestra.run.smithi165.stdout:2103 trunc from 0x8e12e00 to 0x8cda00 2024-09-17T21:46:46.829 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.829 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:46.882 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:46.880+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:46.947 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:46.947 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:46.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:46.965 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:46.963+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:47.066 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:47.063+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:47.103 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:47.103 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:47.111 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:47.195 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:47.193+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:47.258 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:47.259 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:47.267 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:47.308 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:47.306+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:47.369 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:47.367+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:47.415 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:47.416 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:47.425 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:47.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:47.469 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:47.467+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:47.532 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:47.532 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:47.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:47.570 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:47.568+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:47.602 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:47.602 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:47.619 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:47.620 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:47.636 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:47.636 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:47.653 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:47.653 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:47.670 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:47.670 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:47.670 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:47.668+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:47.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:47.687 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.735165231, 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.733140178999999, 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.719154122999999, 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.717192915999998, 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:47.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:47.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:47.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.715159559, 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:47.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.713181640999998, 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:47.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:47.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.713158846999999, 2024-09-17T21:46:47.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:47.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.707249783999998, 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:47.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.726 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.705218725000002, 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:47.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:47.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.705192150999999, 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.731 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:47.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:47.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.704582358, 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:47.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:47.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.704564772000001, 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:47.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:47.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 30.704549815, 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.750 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:47.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.988868326999999, 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:47.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:47.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.988837705999998, 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:47.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:47.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.988813088000001, 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:47.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:47.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:47.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.988791141, 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:47.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:47.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.988769671, 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:47.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:47.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:47.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:47.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.947352867999999, 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:46:47.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.946308408, 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:46:47.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:46:47.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:47.796 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:47.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:46:47.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:47.799 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:47.795+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:47.899 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:47.896+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:47.961 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:47.961 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:47.961 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:47.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:47.999 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:47.997+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:48.100 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:48.098+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:48.116 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:48.116 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:48.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:48.202 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:48.199+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:48.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259283062000002, 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:48.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259260568999998, 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:48.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:48.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259236308999998, 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:48.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259210425999999, 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259185424999998, 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:48.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:48.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259145625999999, 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259129881, 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:48.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259118346000001, 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:48.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:48.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.259108046000001, 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257136803000002, 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:48.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257123655000001, 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:48.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:48.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.286 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257113049000001, 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:48.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:48.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257102711000002, 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:48.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257092704000002, 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:48.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:48.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257082752999999, 2024-09-17T21:46:48.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:48.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.257067445000001, 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:48.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.317 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.255091558, 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:48.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.255080602, 2024-09-17T21:46:48.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:48.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.255068113, 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:48.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.255057650000001, 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:48.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:48.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:48.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:48.336 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:48.338 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:48.335+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:48.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:48.438 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:48.436+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:48.447 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:48.447 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:48.447 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:48.456 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:48.539 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:48.535+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:48.603 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:48.603 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:48.603 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:48.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:48.640 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:48.637+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.433733517999997, 2024-09-17T21:46:48.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:48.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:48.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:48.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.411806714000001, 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:48.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:48.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.732 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.060418677999998, 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:48.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:48.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:359 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24440832~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555489559999998, 2024-09-17T21:46:48.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031798629999999998, 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:48.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161038+0000", 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161039+0000", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4e-07 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161042+0000", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1920000000000001e-06 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161045+0000", 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.548e-06 2024-09-17T21:46:48.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164121+0000", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030756540000000001 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164164+0000", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3234e-05 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164218+0000", 2024-09-17T21:46:48.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4088000000000001e-05 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555479824000003, 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:48.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555470213, 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:48.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555459382000002, 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:48.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:48.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555445790999997, 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:48.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.555435441, 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:48.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.550323063999997, 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:48.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:48.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.545996369000001, 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:48.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.545857159999997, 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:48.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.545840169999998, 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:48.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.545819172000002, 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.540973018000003, 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:48.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:48.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.540960609000003, 2024-09-17T21:46:48.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:48.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.540950277, 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:48.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.033706380999998, 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:48.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:48.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:48.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:48.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.001431405999998, 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:48.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:48.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.001192637999999, 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.819 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:48.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:48.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:48.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:48.824 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:48.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:48.827 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:46:48.827 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:48.822+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:48.834 INFO:teuthology.orchestra.run.smithi165.stdout:2106 read 0x4c6e00 thru 0x4d21bd (0xb3be bytes) 2024-09-17T21:46:48.835 INFO:teuthology.orchestra.run.smithi165.stdout:2107 write 0x71ba00 thru 0x7293ae (0xd9af bytes) 2024-09-17T21:46:48.855 INFO:teuthology.orchestra.run.smithi165.stdout:2108 write 0x294d800 thru 0x294e136 (0x937 bytes) 2024-09-17T21:46:48.856 INFO:teuthology.orchestra.run.smithi165.stdout:2109 punch from 0x5cb200 to 0x5d6200, (0xb000 bytes) 2024-09-17T21:46:48.885 INFO:teuthology.orchestra.run.smithi165.stdout:2110 write 0xd6c4c00 thru 0xd6d435c (0xf75d bytes) 2024-09-17T21:46:48.886 INFO:teuthology.orchestra.run.smithi165.stdout:2111 punch from 0xb348200 to 0xb34a600, (0x2400 bytes) 2024-09-17T21:46:48.888 INFO:teuthology.orchestra.run.smithi165.stdout:2112 punch from 0xaee8e00 to 0xaef2600, (0x9800 bytes) 2024-09-17T21:46:48.888 INFO:teuthology.orchestra.run.smithi165.stdout:2114 punch from 0xed0200 to 0xed7c00, (0x7a00 bytes) 2024-09-17T21:46:48.888 INFO:teuthology.orchestra.run.smithi165.stdout:2115 write 0x8742000 thru 0x8743484 (0x1485 bytes) 2024-09-17T21:46:48.889 INFO:teuthology.orchestra.run.smithi165.stdout:2117 read 0xae59e00 thru 0xae5aeb1 (0x10b2 bytes) 2024-09-17T21:46:48.896 INFO:teuthology.orchestra.run.smithi165.stdout:2118 trunc from 0xd6d4400 to 0x7501600 2024-09-17T21:46:48.927 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:48.923+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:48.933 INFO:teuthology.orchestra.run.smithi165.stdout:2119 write 0x9820800 thru 0x9827457 (0x6c58 bytes) 2024-09-17T21:46:48.937 INFO:teuthology.orchestra.run.smithi165.stdout:2124 trunc from 0x9827600 to 0x647b800 2024-09-17T21:46:48.967 INFO:teuthology.orchestra.run.smithi165.stdout:2125 write 0x675c000 thru 0x676325c (0x725d bytes) 2024-09-17T21:46:48.988 INFO:teuthology.orchestra.run.smithi165.stdout:2126 write 0xc4da600 thru 0xc4dbc32 (0x1633 bytes) 2024-09-17T21:46:48.988 INFO:teuthology.orchestra.run.smithi165.stdout:2129 read 0xa236a00 thru 0xa23b2f7 (0x48f8 bytes) 2024-09-17T21:46:48.998 INFO:teuthology.orchestra.run.smithi165.stdout:2130 trunc from 0xc4dbe00 to 0x4664e00 2024-09-17T21:46:49.003 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:49.003 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:49.003 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:49.027 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:49.024+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:49.040 INFO:teuthology.orchestra.run.smithi165.stdout:2131 write 0xe42ea00 thru 0xe43472e (0x5d2f bytes) 2024-09-17T21:46:49.041 INFO:teuthology.orchestra.run.smithi165.stdout:2132 write 0x7c6ae00 thru 0x7c6cfeb (0x21ec bytes) 2024-09-17T21:46:49.041 INFO:teuthology.orchestra.run.smithi165.stdout:2134 write 0xb2fe200 thru 0xb30a8d6 (0xc6d7 bytes) 2024-09-17T21:46:49.041 INFO:teuthology.orchestra.run.smithi165.stdout:2136 write 0x6702800 thru 0x670a38e (0x7b8f bytes) 2024-09-17T21:46:49.041 INFO:teuthology.orchestra.run.smithi165.stdout:2137 write 0x163a400 thru 0x1646ea2 (0xcaa3 bytes) 2024-09-17T21:46:49.041 INFO:teuthology.orchestra.run.smithi165.stdout:2138 punch from 0x3825e00 to 0x382f600, (0x9800 bytes) 2024-09-17T21:46:49.042 INFO:teuthology.orchestra.run.smithi165.stdout:2139 clone 17 order 22 su 524288 sc 10 2024-09-17T21:46:49.128 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:49.124+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:49.141 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:49.141 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:49.141 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:49.229 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:49.226+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:49.259 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352346719000003, 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352336418, 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352320343999999, 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352309130999998, 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352291692999998, 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352280823000001, 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.352085946000003, 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.350381276, 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.350370638000001, 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.350360545000001, 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.350350425999999, 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:49.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.348220196, 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:49.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.348210465000001, 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:49.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.347427105999998, 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:49.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:49.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.347411536000003, 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:49.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.347318633999997, 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.346824015000003, 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.346679969999997, 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.345994533000002, 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.345632143000003, 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:49.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:49.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.355 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.357 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:49.357 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:49.353+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:49.457 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:49.454+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:49.558 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:49.555+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:49.619 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:49.619 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:49.619 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:49.659 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:49.656+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:49.736 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.923467490999997, 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:49.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:49.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.913816527000002, 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:46:49.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:910 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.913091315999999, 2024-09-17T21:46:49.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082095099999999997, 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 910 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:49.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820301+0000", 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820306+0000", 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7180000000000004e-06 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820601+0000", 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029544599999999999 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820609+0000", 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821192+0000", 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058298099999999999 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821301+0000", 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109826 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.828510+0000", 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072084130000000003 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2005 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.901199899000002, 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076004749999999998, 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2005 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832195+0000", 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.339e-06 2024-09-17T21:46:49.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832201+0000", 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832206+0000", 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3090000000000003e-06 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839559+0000", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073530169999999999 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839641+0000", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1210000000000003e-05 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839792+0000", 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015160099999999999 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2006 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.901123933000001, 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083420109999999999, 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2006 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832272+0000", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832281+0000", 2024-09-17T21:46:49.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2909999999999996e-06 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832288+0000", 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840364+0000", 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080765279999999995 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840441+0000", 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7028999999999997e-05 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840610+0000", 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016856299999999999 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2033 3.8 3:109bf8c5:::rbd_data.11121f8e87bb.000000000000001c:8 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.900948178, 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077497670000000003, 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2033 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832444+0000", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832446+0000", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832453+0000", 2024-09-17T21:46:49.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832457+0000", 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840020+0000", 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075630999999999997 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840101+0000", 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0913000000000003e-05 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840193+0000", 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1823000000000005e-05 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.829870658000001, 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:46:49.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.828408523, 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:46:49.774 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:46:49.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:46:49.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825797404999999, 2024-09-17T21:46:49.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:49.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:46:49.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:46:49.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4125 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:49.786 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825069418999998, 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098331159999999994, 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4125 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908323+0000", 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908327+0000", 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908334+0000", 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4880000000000001e-06 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908342+0000", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8909999999999995e-06 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914269+0000", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059269379999999997 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914325+0000", 2024-09-17T21:46:49.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6508000000000002e-05 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914456+0000", 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130889 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918056+0000", 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035999629999999999 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918075+0000", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8549000000000001e-05 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918119+0000", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4535e-05 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918155+0000", 2024-09-17T21:46:49.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5954000000000002e-05 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.761682948000001, 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:46:49.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:46:49.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:49.799 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.760954726000001, 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:46:49.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:46:49.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:46:49.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.427377590000001, 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:46:49.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:46:49.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:46:49.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.860619088, 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:49.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:46:49.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.853096946000001, 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.816 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:350 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 5242880~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.886202688000001, 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077978400000000003, 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 350 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847191+0000", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0679999999999999e-06 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847203+0000", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1554e-05 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847206+0000", 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-06 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853739+0000", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065326860000000002 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853817+0000", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8072999999999996e-05 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.854987+0000", 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011696720000000001 2024-09-17T21:46:49.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:351 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 16777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.88617704, 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080135269999999995, 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:49.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 351 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847217+0000", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.668e-06 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847225+0000", 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T21:46:49.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847229+0000", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5700000000000001e-06 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855031+0000", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078027970000000002 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855115+0000", 2024-09-17T21:46:49.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3493999999999997e-05 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855228+0000", 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001133 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:358 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 6291456~819200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.886127140999999, 2024-09-17T21:46:49.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095067050000000007, 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 358 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847267+0000", 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.238e-06 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847272+0000", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0509999999999996e-06 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847275+0000", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1209999999999998e-06 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855269+0000", 2024-09-17T21:46:49.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079939260000000002 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855350+0000", 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0716000000000004e-05 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856771+0000", 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014212719999999999 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:359 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.886097113, 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096101729999999996, 2024-09-17T21:46:49.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:49.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847297+0000", 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847302+0000", 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847305+0000", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.072e-06 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856799+0000", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0094936829999999993 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856847+0000", 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7970000000000003e-05 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856905+0000", 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7819000000000003e-05 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4822.0:730 3.d e51/21 3:b2db4994:::rbd_data.111223014882.0000000000000000:head v 51'272, mlcod=51'270)", 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:49.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.697873496, 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096314799999999996, 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:49.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035519+0000", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.44e-07 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039382+0000", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038629329999999998 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039396+0000", 2024-09-17T21:46:49.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4079e-05 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039402+0000", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039448+0000", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100999999999998e-05 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039463+0000", 2024-09-17T21:46:49.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5537e-05 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045104+0000", 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005640305 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045150+0000", 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6103000000000002e-05 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:49.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:49.846 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:49.846 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:49.848 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:49.848 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:49.845+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:49.947 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:49.944+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:49.960 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:49.960 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:49.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4835.0:1837 3.3 3.58f449c3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:49.802810+0000", 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.155988973, 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.156008651, 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4835", 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1239578343", 2024-09-17T21:46:49.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1837 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802810+0000", 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802810+0000", 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802812+0000", 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.782e-06 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802817+0000", 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6189999999999999e-06 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:49.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802829+0000", 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1822e-05 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802835+0000", 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6879999999999996e-06 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802878+0000", 2024-09-17T21:46:49.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3665999999999998e-05 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:49.802886+0000", 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8450000000000003e-06 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:46:49.965 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:49.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:50.010 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:50.010 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:50.027 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:50.027 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:50.043 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:50.043 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:50.048 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:50.046+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:50.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:50.060 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:50.077 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:50.077 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:50.093 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:50.094 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:50.118 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.118 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:50.170 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:50.167+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:50.234 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.234 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.245 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:50.250 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:50.248+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:50.351 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:50.348+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:50.392 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.392 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:50.484 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:50.481+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:50.547 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.547 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.552 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:50.549+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:50.555 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:50.653 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:50.650+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:50.703 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.703 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.712 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:50.755 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:50.751+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:50.828 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:46:50.868 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.868 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.869 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone16 from 0xe434800 (overlap 0x8cda00) to 0x548200 2024-09-17T21:46:50.872 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:50.870+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:50.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:50.957 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:50.955+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:50.989 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:50.989 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:50.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:51.058 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:51.054+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:51.110 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.110 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.119 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:51.159 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:51.156+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:51.168 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-17T21:46:51.234 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.235 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:51.288 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:51.286+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:51.361 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:51.358+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:51.361 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.361 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:51.462 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:51.460+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:51.533 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.534 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:51.562 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:51.560+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:51.663 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:51.661+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.690 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.699 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:51.742 INFO:teuthology.orchestra.run.smithi165.stdout:2140 write 0x7bba000 thru 0x7bc4b72 (0xab73 bytes) 2024-09-17T21:46:51.743 INFO:teuthology.orchestra.run.smithi165.stdout:2143 read 0xdb05a00 thru 0xdb14fde (0xf5df bytes) 2024-09-17T21:46:51.743 INFO:teuthology.orchestra.run.smithi165.stdout:2144 read 0xa681200 thru 0xa68d38d (0xc18e bytes) 2024-09-17T21:46:51.744 INFO:teuthology.orchestra.run.smithi165.stdout:2145 write 0x97c9200 thru 0x97d7726 (0xe527 bytes) 2024-09-17T21:46:51.756 INFO:teuthology.orchestra.run.smithi165.stdout:2146 trunc from 0xe434800 to 0x4d3c000 2024-09-17T21:46:51.783 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:51.781+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:51.789 INFO:teuthology.orchestra.run.smithi165.stdout:2147 trunc from 0x4d3c000 to 0x946c000 2024-09-17T21:46:51.797 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5] out_osds: [0, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:51.797 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:51.797 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-17T21:46:51.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T21:46:51.810 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:46:51.814 INFO:teuthology.orchestra.run.smithi165.stdout:2148 write 0xe0b4000 thru 0xe0b8a4b (0x4a4c bytes) 2024-09-17T21:46:51.815 INFO:teuthology.orchestra.run.smithi165.stdout:2149 write 0x2420c00 thru 0x2423e06 (0x3207 bytes) 2024-09-17T21:46:51.818 INFO:teuthology.orchestra.run.smithi165.stdout:2151 trunc from 0xe0b8c00 to 0xbfbf400 2024-09-17T21:46:51.837 INFO:teuthology.orchestra.run.smithi165.stdout:2152 read 0x7d80800 thru 0x7d89f5b (0x975c bytes) 2024-09-17T21:46:51.838 INFO:teuthology.orchestra.run.smithi165.stdout:2154 write 0x3d3000 thru 0x3e08a1 (0xd8a2 bytes) 2024-09-17T21:46:51.838 INFO:teuthology.orchestra.run.smithi165.stdout:2155 punch from 0xdefe00 to 0xdf3a00, (0x3c00 bytes) 2024-09-17T21:46:51.839 INFO:teuthology.orchestra.run.smithi165.stdout:2156 read 0x512de00 thru 0x5136940 (0x8b41 bytes) 2024-09-17T21:46:51.839 INFO:teuthology.orchestra.run.smithi165.stdout:2159 write 0x53fac00 thru 0x540ab4d (0xff4e bytes) 2024-09-17T21:46:51.846 INFO:teuthology.orchestra.run.smithi165.stdout:2160 trunc from 0xbfbf400 to 0x631e600 2024-09-17T21:46:51.864 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:51.864 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:51.865 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:51.862+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:51.870 INFO:teuthology.orchestra.run.smithi165.stdout:2161 trunc from 0x631e600 to 0xe763a00 2024-09-17T21:46:51.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:51.878 INFO:teuthology.orchestra.run.smithi165.stdout:2162 read 0x45be600 thru 0x45c90ae (0xaaaf bytes) 2024-09-17T21:46:51.884 INFO:teuthology.orchestra.run.smithi165.stdout:2164 write 0x23ba200 thru 0x23bcec8 (0x2cc9 bytes) 2024-09-17T21:46:51.891 INFO:teuthology.orchestra.run.smithi165.stdout:2165 trunc from 0xe763a00 to 0x83c5800 2024-09-17T21:46:51.908 INFO:teuthology.orchestra.run.smithi165.stdout:2169 write 0x81be800 thru 0x81c1c20 (0x3421 bytes) 2024-09-17T21:46:51.909 INFO:teuthology.orchestra.run.smithi165.stdout:2170 write 0x6459000 thru 0x6461c90 (0x8c91 bytes) 2024-09-17T21:46:51.909 INFO:teuthology.orchestra.run.smithi165.stdout:2171 punch from 0x1382600 to 0x1388200, (0x5c00 bytes) 2024-09-17T21:46:51.914 INFO:teuthology.orchestra.run.smithi165.stdout:2172 trunc from 0x83c5800 to 0xcbb6e00 2024-09-17T21:46:51.928 INFO:teuthology.orchestra.run.smithi165.stdout:2173 trunc from 0xcbb6e00 to 0xbaaee00 2024-09-17T21:46:51.937 INFO:teuthology.orchestra.run.smithi165.stdout:2174 read 0x6028600 thru 0x6035c50 (0xd651 bytes) 2024-09-17T21:46:51.938 INFO:teuthology.orchestra.run.smithi165.stdout:2177 write 0x54bfc00 thru 0x54cfa12 (0xfe13 bytes) 2024-09-17T21:46:51.949 INFO:teuthology.orchestra.run.smithi165.stdout:2178 trunc from 0xbaaee00 to 0x2f38c00 2024-09-17T21:46:51.965 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:51.962+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:51.995 INFO:teuthology.orchestra.run.smithi165.stdout:2180 write 0xea8b400 thru 0xea97810 (0xc411 bytes) 2024-09-17T21:46:51.995 INFO:teuthology.orchestra.run.smithi165.stdout:2181 read 0x685e00 thru 0x68d4ad (0x76ae bytes) 2024-09-17T21:46:51.996 INFO:teuthology.orchestra.run.smithi165.stdout:2182 punch from 0x636da00 to 0x637aa00, (0xd000 bytes) 2024-09-17T21:46:51.996 INFO:teuthology.orchestra.run.smithi165.stdout:2183 punch from 0xdd73c00 to 0xdd7ce00, (0x9200 bytes) 2024-09-17T21:46:51.996 INFO:teuthology.orchestra.run.smithi165.stdout:2184 write 0x47b5800 thru 0x47b5829 (0x2a bytes) 2024-09-17T21:46:51.997 INFO:teuthology.orchestra.run.smithi165.stdout:2186 read 0x96a4000 thru 0x96ab9e7 (0x79e8 bytes) 2024-09-17T21:46:52.006 INFO:teuthology.orchestra.run.smithi165.stdout:2188 trunc from 0xea97a00 to 0x6b76a00 2024-09-17T21:46:52.021 INFO:teuthology.orchestra.run.smithi165.stdout:2189 read 0x530f800 thru 0x5311de0 (0x25e1 bytes) 2024-09-17T21:46:52.030 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:52.031 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:52.031 INFO:teuthology.orchestra.run.smithi165.stdout:2190 write 0x8312400 thru 0x8321243 (0xee44 bytes) 2024-09-17T21:46:52.032 INFO:teuthology.orchestra.run.smithi165.stdout:2191 punch from 0x15ef000 to 0x15f3200, (0x4200 bytes) 2024-09-17T21:46:52.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:52.041 INFO:teuthology.orchestra.run.smithi165.stdout:2193 trunc from 0x8321400 to 0x17cd000 2024-09-17T21:46:52.083 INFO:teuthology.orchestra.run.smithi165.stdout:2194 write 0x7ea0e00 thru 0x7eaa73e (0x993f bytes) 2024-09-17T21:46:52.083 INFO:teuthology.orchestra.run.smithi165.stdout:2196 read 0x24c3a00 thru 0x24ca630 (0x6c31 bytes) 2024-09-17T21:46:52.084 INFO:teuthology.orchestra.run.smithi165.stdout:2197 write 0x378f000 thru 0x37967bf (0x77c0 bytes) 2024-09-17T21:46:52.084 INFO:teuthology.orchestra.run.smithi165.stdout:2198 punch from 0x7018a00 to 0x701ce00, (0x4400 bytes) 2024-09-17T21:46:52.084 INFO:teuthology.orchestra.run.smithi165.stdout:2199 punch from 0x72fb000 to 0x7306a00, (0xba00 bytes) 2024-09-17T21:46:52.084 INFO:teuthology.orchestra.run.smithi165.stdout:2201 read 0x1c9dc00 thru 0x1c9ffb8 (0x23b9 bytes) 2024-09-17T21:46:52.085 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:52.083+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:52.097 INFO:teuthology.orchestra.run.smithi165.stdout:2202 trunc from 0x7eaa800 to 0xe643000 2024-09-17T21:46:52.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:52.107 INFO:teuthology.orchestra.run.smithi165.stdout:2203 punch from 0xa7d7600 to 0xa7df600, (0x8000 bytes) 2024-09-17T21:46:52.108 INFO:teuthology.orchestra.run.smithi165.stdout:2204 punch from 0x3247200 to 0x324e200, (0x7000 bytes) 2024-09-17T21:46:52.113 INFO:teuthology.orchestra.run.smithi165.stdout:2206 trunc from 0xe643000 to 0x9d16e00 2024-09-17T21:46:52.127 INFO:teuthology.orchestra.run.smithi165.stdout:2208 punch from 0x6af4200 to 0x6afc600, (0x8400 bytes) 2024-09-17T21:46:52.129 INFO:teuthology.orchestra.run.smithi165.stdout:2211 write 0x6bc1400 thru 0x6bce3ae (0xcfaf bytes) 2024-09-17T21:46:52.129 INFO:teuthology.orchestra.run.smithi165.stdout:2212 read 0x2fe3400 thru 0x2fed64b (0xa24c bytes) 2024-09-17T21:46:52.129 INFO:teuthology.orchestra.run.smithi165.stdout:2216 read 0x20de600 thru 0x20e9572 (0xaf73 bytes) 2024-09-17T21:46:52.131 INFO:teuthology.orchestra.run.smithi165.stdout:2217 read 0x64b4800 thru 0x64b7399 (0x2b9a bytes) 2024-09-17T21:46:52.131 INFO:teuthology.orchestra.run.smithi165.stdout:2218 read 0x40b2200 thru 0x40bbd1a (0x9b1b bytes) 2024-09-17T21:46:52.131 INFO:teuthology.orchestra.run.smithi165.stdout:2220 write 0x9422200 thru 0x942f9c0 (0xd7c1 bytes) 2024-09-17T21:46:52.132 INFO:teuthology.orchestra.run.smithi165.stdout:2221 read 0x1794400 thru 0x179efc5 (0xabc6 bytes) 2024-09-17T21:46:52.132 INFO:teuthology.orchestra.run.smithi165.stdout:2223 read 0x96d4a00 thru 0x96d7d3b (0x333c bytes) 2024-09-17T21:46:52.132 INFO:teuthology.orchestra.run.smithi165.stdout:2229 punch from 0x8948a00 to 0x8957400, (0xea00 bytes) 2024-09-17T21:46:52.132 INFO:teuthology.orchestra.run.smithi165.stdout:2232 write 0x54af200 thru 0x54bbf87 (0xcd88 bytes) 2024-09-17T21:46:52.133 INFO:teuthology.orchestra.run.smithi165.stdout:2233 write 0x2b99a00 thru 0x2ba1b37 (0x8138 bytes) 2024-09-17T21:46:52.133 INFO:teuthology.orchestra.run.smithi165.stdout:2234 read 0x985c000 thru 0x9861129 (0x512a bytes) 2024-09-17T21:46:52.133 INFO:teuthology.orchestra.run.smithi165.stdout:2238 write 0x14fc800 thru 0x15089b6 (0xc1b7 bytes) 2024-09-17T21:46:52.133 INFO:teuthology.orchestra.run.smithi165.stdout:2240 trunc from 0x9d16e00 to 0x9f87400 2024-09-17T21:46:52.149 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:52.147+0000 7f0c28614640 -1 mon.a@0(leader).osd e64 definitely_dead 0 2024-09-17T21:46:52.152 INFO:teuthology.orchestra.run.smithi165.stdout:2242 trunc from 0x9f87400 to 0x60cd200 2024-09-17T21:46:52.166 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:52.164+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:52.167 INFO:teuthology.orchestra.run.smithi165.stdout:2243 read 0xbc2200 thru 0xbc8ef3 (0x6cf4 bytes) 2024-09-17T21:46:52.168 INFO:teuthology.orchestra.run.smithi165.stdout:2246 write 0x3215600 thru 0x321f87c (0xa27d bytes) 2024-09-17T21:46:52.169 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:52.169 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:52.169 INFO:teuthology.orchestra.run.smithi165.stdout:2247 punch from 0x3f4b600 to 0x3f5a800, (0xf200 bytes) 2024-09-17T21:46:52.170 INFO:teuthology.orchestra.run.smithi165.stdout:2248 write 0x5602c00 thru 0x560be55 (0x9256 bytes) 2024-09-17T21:46:52.170 INFO:teuthology.orchestra.run.smithi165.stdout:2252 read 0x2361200 thru 0x2362fb9 (0x1dba bytes) 2024-09-17T21:46:52.176 INFO:teuthology.orchestra.run.smithi165.stdout:2258 trunc from 0x60cd200 to 0xb94c400 2024-09-17T21:46:52.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:52.198 INFO:teuthology.orchestra.run.smithi165.stdout:2259 write 0xc21b600 thru 0xc221277 (0x5c78 bytes) 2024-09-17T21:46:52.199 INFO:teuthology.orchestra.run.smithi165.stdout:2260 write 0x5121600 thru 0x5126de1 (0x57e2 bytes) 2024-09-17T21:46:52.199 INFO:teuthology.orchestra.run.smithi165.stdout:2261 punch from 0x309b200 to 0x30a8e00, (0xdc00 bytes) 2024-09-17T21:46:52.200 INFO:teuthology.orchestra.run.smithi165.stdout:2262 punch from 0x7229000 to 0x722f000, (0x6000 bytes) 2024-09-17T21:46:52.200 INFO:teuthology.orchestra.run.smithi165.stdout:2265 write 0xa81c400 thru 0xa828073 (0xbc74 bytes) 2024-09-17T21:46:52.212 INFO:teuthology.orchestra.run.smithi165.stdout:2268 trunc from 0xc221400 to 0x214b400 2024-09-17T21:46:52.248 INFO:teuthology.orchestra.run.smithi165.stdout:2270 write 0x7ac5000 thru 0x7acb7ce (0x67cf bytes) 2024-09-17T21:46:52.251 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:52.251 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:52.267 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:52.265+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:52.270 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:52.270 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:52.287 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:52.288 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:52.304 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:52.305 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:52.321 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:52.322 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:52.333 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:52.333 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:52.333 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:52.338 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:52.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:52.391 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:52.389+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:52.470 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:52.467+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:52.497 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.684121978, 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:52.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:52.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.674471014000002, 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:46:52.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:910 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.673745802999999, 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082095099999999997, 2024-09-17T21:46:52.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 910 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820301+0000", 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820306+0000", 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7180000000000004e-06 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820601+0000", 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029544599999999999 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820609+0000", 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:52.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821192+0000", 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058298099999999999 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821301+0000", 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109826 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.828510+0000", 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072084130000000003 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2005 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.661854386000002, 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076004749999999998, 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2005 2024-09-17T21:46:52.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832195+0000", 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.339e-06 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832201+0000", 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832206+0000", 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3090000000000003e-06 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839559+0000", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073530169999999999 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839641+0000", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1210000000000003e-05 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839792+0000", 2024-09-17T21:46:52.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015160099999999999 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2006 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.661778420000001, 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083420109999999999, 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2006 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:52.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832272+0000", 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832281+0000", 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2909999999999996e-06 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832288+0000", 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840364+0000", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080765279999999995 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840441+0000", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7028999999999997e-05 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840610+0000", 2024-09-17T21:46:52.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016856299999999999 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2033 3.8 3:109bf8c5:::rbd_data.11121f8e87bb.000000000000001c:8 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.661602665, 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077497670000000003, 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2033 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832444+0000", 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832446+0000", 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832453+0000", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832457+0000", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840020+0000", 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075630999999999997 2024-09-17T21:46:52.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840101+0000", 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0913000000000003e-05 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840193+0000", 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1823000000000005e-05 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.590525145000001, 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:46:52.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.58906301, 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:46:52.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:46:52.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.586451891999999, 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.546 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:46:52.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:52.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4125 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.585723905999998, 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098331159999999994, 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4125 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908323+0000", 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908327+0000", 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908334+0000", 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4880000000000001e-06 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908342+0000", 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8909999999999995e-06 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914269+0000", 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059269379999999997 2024-09-17T21:46:52.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914325+0000", 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6508000000000002e-05 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914456+0000", 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130889 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918056+0000", 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035999629999999999 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918075+0000", 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8549000000000001e-05 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918119+0000", 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4535e-05 2024-09-17T21:46:52.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918155+0000", 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5954000000000002e-05 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:52.559 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.522337435000001, 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:46:52.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:46:52.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.521609213000001, 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:46:52.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.188032076999999, 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.621273575, 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:52.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:46:52.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.613751433000001, 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:46:52.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:46:52.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:350 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 5242880~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.646857175000001, 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077978400000000003, 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 350 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:52.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847191+0000", 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0679999999999999e-06 2024-09-17T21:46:52.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847203+0000", 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1554e-05 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847206+0000", 2024-09-17T21:46:52.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-06 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853739+0000", 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065326860000000002 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853817+0000", 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8072999999999996e-05 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.854987+0000", 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011696720000000001 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:351 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 16777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.646831527, 2024-09-17T21:46:52.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080135269999999995, 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 351 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847217+0000", 2024-09-17T21:46:52.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.668e-06 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847225+0000", 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847229+0000", 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5700000000000001e-06 2024-09-17T21:46:52.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855031+0000", 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078027970000000002 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855115+0000", 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3493999999999997e-05 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855228+0000", 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001133 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:358 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 6291456~819200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.646781627999999, 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095067050000000007, 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 358 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847267+0000", 2024-09-17T21:46:52.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.238e-06 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847272+0000", 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0509999999999996e-06 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847275+0000", 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1209999999999998e-06 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855269+0000", 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079939260000000002 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855350+0000", 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0716000000000004e-05 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856771+0000", 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014212719999999999 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:359 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.646751600000002, 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096101729999999996, 2024-09-17T21:46:52.606 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847297+0000", 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847302+0000", 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847305+0000", 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.072e-06 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856799+0000", 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0094936829999999993 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856847+0000", 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7970000000000003e-05 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856905+0000", 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7819000000000003e-05 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4822.0:730 3.d e51/21 3:b2db4994:::rbd_data.111223014882.0000000000000000:head v 51'272, mlcod=51'270)", 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.458527983, 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096314799999999996, 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:52.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035519+0000", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.44e-07 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039382+0000", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038629329999999998 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039396+0000", 2024-09-17T21:46:52.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4079e-05 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039402+0000", 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039448+0000", 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100999999999998e-05 2024-09-17T21:46:52.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039463+0000", 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5537e-05 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045104+0000", 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005640305 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:52.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045150+0000", 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6103000000000002e-05 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:52.616 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:52.618 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:52.615+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:52.619 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2272 write 0xd08de00 thru 0xd092c17 (0x4e18 bytes) 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2275 read 0x9b14400 thru 0x9b20fac (0xcbad bytes) 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2278 write 0xc6f3e00 thru 0xc6ff040 (0xb241 bytes) 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2279 punch from 0xb009000 to 0xb017200, (0xe200 bytes) 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2280 trunc from 0xd092e00 to 0xda8ce00 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2281 read 0x26ea400 thru 0x26ed856 (0x3457 bytes) 2024-09-17T21:46:52.620 INFO:teuthology.orchestra.run.smithi165.stdout:2282 trunc from 0xda8ce00 to 0x3bf3a00 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2283 trunc from 0x3bf3a00 to 0xb45cc00 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2287 punch from 0x493ee00 to 0x4946000, (0x7200 bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2288 punch from 0x204f000 to 0x205ac00, (0xbc00 bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2289 write 0xb8bd600 thru 0xb8bf1e1 (0x1be2 bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2290 read 0x492e200 thru 0x493b65b (0xd45c bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2292 write 0x720f200 thru 0x72128a9 (0x36aa bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2294 write 0xa070e00 thru 0xa075ab5 (0x4cb6 bytes) 2024-09-17T21:46:52.621 INFO:teuthology.orchestra.run.smithi165.stdout:2296 trunc from 0xb8bf200 to 0xc895000 2024-09-17T21:46:52.639 INFO:teuthology.orchestra.run.smithi165.stdout:2297 write 0xec6f400 thru 0xec71022 (0x1c23 bytes) 2024-09-17T21:46:52.644 INFO:teuthology.orchestra.run.smithi165.stdout:2298 write 0x23f8400 thru 0x2403f11 (0xbb12 bytes) 2024-09-17T21:46:52.645 INFO:teuthology.orchestra.run.smithi165.stdout:2300 read 0xc4be400 thru 0xc4c538f (0x6f90 bytes) 2024-09-17T21:46:52.645 INFO:teuthology.orchestra.run.smithi165.stdout:2303 clone 18 order 24 su 65536 sc 4 2024-09-17T21:46:52.719 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:52.716+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:52.732 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:52.732 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:52.732 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:52.732 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:52.740 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.0. 2024-09-17T21:46:52.819 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:52.816+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:52.831 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-17T21:46:52.895 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:52.896 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:52.896 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:52.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:52.921 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:52.918+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.850042200999994, 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.013 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:53.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:53.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:53.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.189056912000002, 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.171485521000001, 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:46:53.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:46:53.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:53.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.171451827999999, 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:46:53.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168610524999998, 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:46:53.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:46:53.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168533214, 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:46:53.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168510821000002, 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:46:53.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:46:53.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168489699999999, 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:46:53.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.053 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168468956000002, 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:46:53.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168054311999999, 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:46:53.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.168019121, 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:46:53.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:46:53.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.167127657999998, 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:46:53.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:46:53.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.166761589, 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:46:53.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.113454371, 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:46:53.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:46:53.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:53.081 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.113418598999999, 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:46:53.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.109280226999999, 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:46:53.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.104086506000002, 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:46:53.092 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:46:53.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:46:53.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:46:53.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.711579913000001, 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:46:53.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.140280180000001, 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:46:53.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:53.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:46:53.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:46:53.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:46:53.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.134732817, 2024-09-17T21:46:53.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:46:53.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:46:53.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.120 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:53.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:53.122 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:53.119+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:53.222 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:53.220+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:53.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:53.269 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:53.277 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:53.363 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:53.361+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:53.424 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:53.421+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:53.426 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:53.426 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:53.426 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:53.426 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:53.426 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:53.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:53.525 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:53.522+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.629137102999998, 2024-09-17T21:46:53.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:53.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:46:53.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.627112050999997, 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.593 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.613125994999997, 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.598 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.611164787999996, 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:46:53.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.609131431000002, 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:46:53.603 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:46:53.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.607153513, 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:46:53.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.607130718999997, 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:46:53.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.601221656, 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:46:53.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.599190597000003, 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:46:53.622 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:46:53.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:46:53.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.599164023, 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:46:53.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:46:53.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.598554229999998, 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:46:53.633 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:53.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:46:53.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:46:53.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.598536643999999, 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:46:53.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:46:53.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:46:53.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.598521687000002, 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.882840199, 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:46:53.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:46:53.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.882809578, 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:46:53.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:46:53.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.882784960000002, 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:46:53.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:46:53.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.882763013000002, 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:46:53.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:46:53.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:46:53.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:46:53.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.882741543000002, 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:53.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:46:53.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:46:53.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.841324740000001, 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:46:53.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:46:53.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.840280280000002, 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:53.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:46:53.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:53.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:53.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:53.695 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:53.696 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:53.697 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:53.693+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:53.797 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:53.795+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:53.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:46:53.845 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:53.845 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:53.845 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:53.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:53.937 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:53.935+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:53.999 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:53.996+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:54.001 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:54.001 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:54.001 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:54.001 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:54.001 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:54.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:54.100 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:54.098+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:54.158 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:54.158 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:54.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:54.158 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:54.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184172246999999, 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:46:54.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:46:54.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.163 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184149754000003, 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.164 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:46:54.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184125494, 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:46:54.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184099611000001, 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:46:54.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:46:54.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184074610000003, 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:46:54.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184034810999997, 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:46:54.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184019065999998, 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:46:54.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:46:54.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.193 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.184007530999999, 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:46:54.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.183997230999999, 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:46:54.198 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:46:54.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.182025987999999, 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:46:54.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.182012839999999, 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:46:54.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.182002234000002, 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:46:54.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:46:54.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.181991896, 2024-09-17T21:46:54.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:46:54.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.181981888999999, 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:46:54.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:46:54.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.181971937999997, 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:46:54.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:46:54.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.181956630000002, 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:46:54.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.236 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.179980743000002, 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:46:54.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:46:54.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.179969786999997, 2024-09-17T21:46:54.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:54.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:46:54.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.179957297999998, 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.179946835000003, 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:46:54.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:54.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:46:54.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:46:54.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:54.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:54.256 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:54.257 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:54.255+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:54.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:54.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:54.358 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:54.356+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:54.365 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:54.366 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:54.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:54.459 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:54.456+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:54.488 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:54.488 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:54.505 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:54.505 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:54.521 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:54.522 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:54.534 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:54.534 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:54.538 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:54.538 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:54.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:54.556 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:54.557 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:54.560 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:54.558+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:54.573 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:54.654 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:46:54.661 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:54.658+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:54.669 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:54.669 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:54.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:54.762 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:54.760+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:54.837 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone17 from 0xec71200 (overlap 0x17cd000) to 0x0 2024-09-17T21:46:54.842 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:54.842 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:54.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:54.897 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:54.894+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:54.963 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:54.961+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:54.981 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:54.981 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:54.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:55.064 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:55.062+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:55.156 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.156 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.168 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:55.163+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:55.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:55.169 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-17T21:46:55.263 INFO:teuthology.orchestra.run.smithi165.stdout:2305 trunc from 0xec71200 to 0x5f03000 2024-09-17T21:46:55.266 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:55.264+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:55.291 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.291 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:55.367 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:55.365+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:55.424 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.424 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:55.480 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:55.477+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:55.569 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:55.566+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:55.570 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.570 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:55.618 INFO:teuthology.orchestra.run.smithi165.stdout:2315 write 0x7322c00 thru 0x732c9bf (0x9dc0 bytes) 2024-09-17T21:46:55.623 INFO:teuthology.orchestra.run.smithi165.stdout:2316 write 0x5228400 thru 0x5228a19 (0x61a bytes) 2024-09-17T21:46:55.629 INFO:teuthology.orchestra.run.smithi165.stdout:2317 trunc from 0x732ca00 to 0xbe1e400 2024-09-17T21:46:55.641 INFO:teuthology.orchestra.run.smithi165.stdout:2318 read 0x129c400 thru 0x12a1c92 (0x5893 bytes) 2024-09-17T21:46:55.649 INFO:teuthology.orchestra.run.smithi165.stdout:2320 write 0x5542800 thru 0x5544e11 (0x2612 bytes) 2024-09-17T21:46:55.649 INFO:teuthology.orchestra.run.smithi165.stdout:2322 punch from 0x5e8bc00 to 0x5e98800, (0xcc00 bytes) 2024-09-17T21:46:55.650 INFO:teuthology.orchestra.run.smithi165.stdout:2323 write 0x68c6e00 thru 0x68d07da (0x99db bytes) 2024-09-17T21:46:55.652 INFO:teuthology.orchestra.run.smithi165.stdout:2324 write 0x6c1a400 thru 0x6c206e8 (0x62e9 bytes) 2024-09-17T21:46:55.653 INFO:teuthology.orchestra.run.smithi165.stdout:2327 trunc from 0xbe1e400 to 0xbac3800 2024-09-17T21:46:55.670 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:55.667+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:55.684 INFO:teuthology.orchestra.run.smithi165.stdout:2329 write 0xcb3b400 thru 0xcb43165 (0x7d66 bytes) 2024-09-17T21:46:55.691 INFO:teuthology.orchestra.run.smithi165.stdout:2330 read 0xc937000 thru 0xc93b438 (0x4439 bytes) 2024-09-17T21:46:55.691 INFO:teuthology.orchestra.run.smithi165.stdout:2331 punch from 0x4abe400 to 0x4ac7400, (0x9000 bytes) 2024-09-17T21:46:55.692 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.692 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.692 INFO:teuthology.orchestra.run.smithi165.stdout:2334 read 0x56eb800 thru 0x56f3e17 (0x8618 bytes) 2024-09-17T21:46:55.694 INFO:teuthology.orchestra.run.smithi165.stdout:2335 read 0x7eb4600 thru 0x7ebd1a6 (0x8ba7 bytes) 2024-09-17T21:46:55.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:55.708 INFO:teuthology.orchestra.run.smithi165.stdout:2336 trunc from 0xcb43200 to 0xa88200 2024-09-17T21:46:55.771 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:55.768+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:55.792 INFO:teuthology.orchestra.run.smithi165.stdout:2338 trunc from 0xa88200 to 0x8b63400 2024-09-17T21:46:55.801 INFO:teuthology.orchestra.run.smithi165.stdout:2339 punch from 0x3ae600 to 0x3b0c00, (0x2600 bytes) 2024-09-17T21:46:55.805 INFO:teuthology.orchestra.run.smithi165.stdout:2340 punch from 0x7c0d600 to 0x7c0dc00, (0x600 bytes) 2024-09-17T21:46:55.805 INFO:teuthology.orchestra.run.smithi165.stdout:2341 write 0x8940e00 thru 0x8950b05 (0xfd06 bytes) 2024-09-17T21:46:55.814 INFO:teuthology.orchestra.run.smithi165.stdout:2342 trunc from 0x8b63400 to 0xa29800 2024-09-17T21:46:55.825 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.825 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:55.878 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:55.876+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:55.942 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:55.942 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:55.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:55.973 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:55.970+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:56.074 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:56.071+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:56.098 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.098 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.106 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:56.175 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:56.172+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:56.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.254 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:56.307 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:56.305+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:56.370 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.370 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.375 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:56.373+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:56.378 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:56.477 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:56.474+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:56.526 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.526 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:56.578 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:56.576+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:56.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:46:56.641 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.642 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.650 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:56.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:46:56.692 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:56.689+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:56.717 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:56.717 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:56.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:56.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:56.751 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:56.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:56.769 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:56.780 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:56.777+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:56.786 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:56.786 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:56.799 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:56.799 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:56.806 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:56.806 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:56.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:56.881 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:56.878+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:56.893 INFO:teuthology.orchestra.run.smithi165.stdout:2343 write 0xcb2ae00 thru 0xcb3365c (0x885d bytes) 2024-09-17T21:46:56.895 INFO:teuthology.orchestra.run.smithi165.stdout:2345 trunc from 0xcb33800 to 0xde79400 2024-09-17T21:46:56.910 INFO:teuthology.orchestra.run.smithi165.stdout:2347 punch from 0xc398000 to 0xc3a3000, (0xb000 bytes) 2024-09-17T21:46:56.911 INFO:teuthology.orchestra.run.smithi165.stdout:2349 read 0x13e200 thru 0x146272 (0x8073 bytes) 2024-09-17T21:46:56.912 INFO:teuthology.orchestra.run.smithi165.stdout:2350 punch from 0xd283000 to 0xd287800, (0x4800 bytes) 2024-09-17T21:46:56.912 INFO:teuthology.orchestra.run.smithi165.stdout:2351 write 0x98fba00 thru 0x98fd450 (0x1a51 bytes) 2024-09-17T21:46:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:2354 read 0xb2bc00 thru 0xb30ffd (0x53fe bytes) 2024-09-17T21:46:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:2355 punch from 0x5872600 to 0x5873200, (0xc00 bytes) 2024-09-17T21:46:56.913 INFO:teuthology.orchestra.run.smithi165.stdout:2357 write 0x5a4b600 thru 0x5a56e72 (0xb873 bytes) 2024-09-17T21:46:56.924 INFO:teuthology.orchestra.run.smithi165.stdout:2360 trunc from 0xde79400 to 0x5902400 2024-09-17T21:46:56.945 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:56.946 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:56.946 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:56.946 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:56.955 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:56.961 INFO:teuthology.orchestra.run.smithi165.stdout:2361 write 0xb049e00 thru 0xb04e2cf (0x44d0 bytes) 2024-09-17T21:46:56.962 INFO:teuthology.orchestra.run.smithi165.stdout:2362 write 0x8692200 thru 0x86954d3 (0x32d4 bytes) 2024-09-17T21:46:56.962 INFO:teuthology.orchestra.run.smithi165.stdout:2365 write 0x7164200 thru 0x716c262 (0x8063 bytes) 2024-09-17T21:46:56.966 INFO:teuthology.orchestra.run.smithi165.stdout:2367 trunc from 0xb04e400 to 0xe107600 2024-09-17T21:46:56.981 INFO:teuthology.orchestra.run.smithi165.stdout:2369 read 0x19e5c00 thru 0x19ed90d (0x7d0e bytes) 2024-09-17T21:46:56.982 INFO:teuthology.orchestra.run.smithi165.stdout:2370 punch from 0x6e5a800 to 0x6e5b600, (0xe00 bytes) 2024-09-17T21:46:56.983 INFO:teuthology.orchestra.run.smithi165.stdout:2371 read 0x1dd4000 thru 0x1dddd3d (0x9d3e bytes) 2024-09-17T21:46:56.983 INFO:teuthology.orchestra.run.smithi165.stdout:2372 punch from 0x3450600 to 0x345dc00, (0xd600 bytes) 2024-09-17T21:46:56.983 INFO:teuthology.orchestra.run.smithi165.stdout:2375 write 0x8c0c600 thru 0x8c1bad0 (0xf4d1 bytes) 2024-09-17T21:46:56.984 INFO:teuthology.orchestra.run.smithi165.stdout:2376 read 0x95ab800 thru 0x95b6552 (0xad53 bytes) 2024-09-17T21:46:56.985 INFO:teuthology.orchestra.run.smithi165.stdout:2380 read 0x2e9ae00 thru 0x2e9e2b9 (0x34ba bytes) 2024-09-17T21:46:56.985 INFO:teuthology.orchestra.run.smithi165.stdout:2381 write 0x1e7000 thru 0x1edce0 (0x6ce1 bytes) 2024-09-17T21:46:56.996 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:56.993+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:56.997 INFO:teuthology.orchestra.run.smithi165.stdout:2382 write 0xed83e00 thru 0xed8dc4f (0x9e50 bytes) 2024-09-17T21:46:57.016 INFO:teuthology.orchestra.run.smithi165.stdout:2383 trunc from 0xed8de00 to 0x10f9e00 2024-09-17T21:46:57.041 INFO:teuthology.orchestra.run.smithi165.stdout:2384 trunc from 0x10f9e00 to 0x6824200 2024-09-17T21:46:57.049 INFO:teuthology.orchestra.run.smithi165.stdout:2386 write 0x5266400 thru 0x5275c03 (0xf804 bytes) 2024-09-17T21:46:57.069 INFO:teuthology.orchestra.run.smithi165.stdout:2387 write 0xd4e8c00 thru 0xd4e9596 (0x997 bytes) 2024-09-17T21:46:57.070 INFO:teuthology.orchestra.run.smithi165.stdout:2389 read 0x5481800 thru 0x548d4dd (0xbcde bytes) 2024-09-17T21:46:57.070 INFO:teuthology.orchestra.run.smithi165.stdout:2394 write 0x828b400 thru 0x828f5cc (0x41cd bytes) 2024-09-17T21:46:57.074 INFO:teuthology.orchestra.run.smithi165.stdout:2396 write 0x5e4cc00 thru 0x5e4d7cc (0xbcd bytes) 2024-09-17T21:46:57.083 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:57.080+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:57.084 INFO:teuthology.orchestra.run.smithi165.stdout:2397 write 0xebb1400 thru 0xebc0d3e (0xf93f bytes) 2024-09-17T21:46:57.085 INFO:teuthology.orchestra.run.smithi165.stdout:2398 read 0xaed1000 thru 0xaed7475 (0x6476 bytes) 2024-09-17T21:46:57.085 INFO:teuthology.orchestra.run.smithi165.stdout:2399 read 0xdf90a00 thru 0xdf99a7a (0x907b bytes) 2024-09-17T21:46:57.089 INFO:teuthology.orchestra.run.smithi165.stdout:2400 read 0x15ece00 thru 0x15f6773 (0x9974 bytes) 2024-09-17T21:46:57.096 INFO:teuthology.orchestra.run.smithi165.stdout:2401 trunc from 0xebc0e00 to 0x5b04e00 2024-09-17T21:46:57.117 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:57.118 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.127 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:46:57.135 INFO:teuthology.orchestra.run.smithi165.stdout:2405 write 0xdc4ae00 thru 0xdc55fea (0xb1eb bytes) 2024-09-17T21:46:57.141 INFO:teuthology.orchestra.run.smithi165.stdout:2407 write 0x972d600 thru 0x973316c (0x5b6d bytes) 2024-09-17T21:46:57.142 INFO:teuthology.orchestra.run.smithi165.stdout:2408 punch from 0x236ba00 to 0x2372600, (0x6c00 bytes) 2024-09-17T21:46:57.143 INFO:teuthology.orchestra.run.smithi165.stdout:2409 read 0x43cc400 thru 0x43d34dd (0x70de bytes) 2024-09-17T21:46:57.156 INFO:teuthology.orchestra.run.smithi165.stdout:2411 trunc from 0xdc56000 to 0x2acd000 2024-09-17T21:46:57.172 INFO:teuthology.orchestra.run.smithi165.stdout:2413 read 0x2732600 thru 0x2734492 (0x1e93 bytes) 2024-09-17T21:46:57.184 INFO:teuthology.orchestra.run.smithi165.stdout:2415 write 0x2cb0800 thru 0x2cb5053 (0x4854 bytes) 2024-09-17T21:46:57.184 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:57.181+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:57.186 INFO:teuthology.orchestra.run.smithi165.stdout:2417 read 0x1dcea00 thru 0x1dd28e3 (0x3ee4 bytes) 2024-09-17T21:46:57.186 INFO:teuthology.orchestra.run.smithi165.stdout:2418 read 0x1adb000 thru 0x1aeae27 (0xfe28 bytes) 2024-09-17T21:46:57.188 INFO:teuthology.orchestra.run.smithi165.stdout:2420 write 0x175c00 thru 0x17948d (0x388e bytes) 2024-09-17T21:46:57.188 INFO:teuthology.orchestra.run.smithi165.stdout:2421 read 0x1346e00 thru 0x1351b3f (0xad40 bytes) 2024-09-17T21:46:57.188 INFO:teuthology.orchestra.run.smithi165.stdout:2422 trunc from 0x2cb5200 to 0x317fc00 2024-09-17T21:46:57.216 INFO:teuthology.orchestra.run.smithi165.stdout:2424 write 0x97c8000 thru 0x97d5ecd (0xdece bytes) 2024-09-17T21:46:57.216 INFO:teuthology.orchestra.run.smithi165.stdout:2426 read 0x5bf8000 thru 0x5c01b9d (0x9b9e bytes) 2024-09-17T21:46:57.216 INFO:teuthology.orchestra.run.smithi165.stdout:2427 read 0x2b13400 thru 0x2b1cee8 (0x9ae9 bytes) 2024-09-17T21:46:57.217 INFO:teuthology.orchestra.run.smithi165.stdout:2428 punch from 0x10bac00 to 0x10c8400, (0xd800 bytes) 2024-09-17T21:46:57.220 INFO:teuthology.orchestra.run.smithi165.stdout:2429 clone 19 order 24 su 1048576 sc 2 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:57.249 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.962912496000001, 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:46:57.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:46:57.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.940985691999998, 2024-09-17T21:46:57.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:57.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.589597656000002, 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:46:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:46:57.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.084658802, 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:46:57.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.084649190999997, 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:46:57.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:46:57.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.08463836, 2024-09-17T21:46:57.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:57.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:46:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.084624769000001, 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.084614418999998, 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:413 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24571904~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.079502042000001, 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003192187, 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 413 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166205+0000", 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9900000000000001e-07 2024-09-17T21:46:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166211+0000", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166219+0000", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8019999999999998e-06 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.166227+0000", 2024-09-17T21:46:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.109e-06 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169294+0000", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030675339999999998 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169341+0000", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7011000000000001e-05 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.169397+0000", 2024-09-17T21:46:57.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.075175346999998, 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:46:57.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.075036138000002, 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:46:57.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:46:57.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.075019148000003, 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:46:57.306 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:46:57.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.074998149999999, 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:46:57.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:46:57.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.070151996, 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:46:57.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:46:57.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:46:57.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.070139587, 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.070129254999998, 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.325 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:46:57.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:57.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.562885358999999, 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:46:57.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:46:57.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.530610383999999, 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:46:57.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:46:57.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:46:57.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.530371615999996, 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:46:57.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:46:57.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:46:57.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.385662001, 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:46:57.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:46:57.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:46:57.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.351 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:46:57.352 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:57.349+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:57.453 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:57.450+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:57.459 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.459 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:57.459 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:46:57.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:46:57.554 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:57.551+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:57.575 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:57.575 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:46:57.655 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:57.652+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:57.691 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.785015231000003, 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:46:57.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.785004929999999, 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:46:57.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.784988855999998, 2024-09-17T21:46:57.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:46:57.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:46:57.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.784977642999998, 2024-09-17T21:46:57.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:46:57.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.784960204999997, 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.711 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:46:57.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.715 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.784949335, 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.784754458000002, 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:46:57.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:46:57.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:46:57.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:57.724 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.783049788, 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:46:57.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.78303915, 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:46:57.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:46:57.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.783029057, 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:46:57.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:46:57.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.738 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.783018937999998, 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:46:57.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:46:57.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.780888707999999, 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:46:57.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:46:57.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.780878977, 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:46:57.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:46:57.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.780095617999997, 2024-09-17T21:46:57.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:57.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:46:57.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.780080048000002, 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:46:57.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.779987146000003, 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:46:57.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:46:57.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.779492527000002, 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:46:57.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.779348482000003, 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:46:57.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:57.775 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.778663045000002, 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:46:57.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.778300655000002, 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:57.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:46:57.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.784 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:46:57.786 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:57.782+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:57.827 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:46:57.831 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:46:57.831 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:46:57.832 INFO:tasks.thrashosds.thrasher:Setting full ratio to .001 2024-09-17T21:46:57.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .001 2024-09-17T21:46:57.886 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:57.884+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:57.922 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone18 from 0x97d6000 (overlap 0xa29800) to 0x618e00 2024-09-17T21:46:57.946 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4913.0:2585 3.7 3:e26e959f:::rbd_data.11122f545aff.0000000000000003:head [stat out=16b,truncate 1572864] snapc 15=[15] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:57.942726+0000", 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0016795340000000001, 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016996330000000001, 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4913", 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2928674605", 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2585 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942726+0000", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942726+0000", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942727+0000", 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.037e-06 2024-09-17T21:46:57.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942733+0000", 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5130000000000004e-06 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942745+0000", 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1929e-05 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:57.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942750+0000", 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7400000000000001e-06 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942811+0000", 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0668000000000003e-05 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.942908+0000", 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6469000000000001e-05 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:57.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:57.943095+0000", 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018785399999999999 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:46:57.951 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:57.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:46:58.001 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:57.998+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:58.072 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:46:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:46:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:46:58.072 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:58.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:46:58.123 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:58.119+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:58.189 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:58.186+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.421517281, 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:46:58.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:46:58.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.411866317000001, 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:46:58.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:910 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.411141105999999, 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082095099999999997, 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 910 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820300+0000", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820301+0000", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9299999999999997e-07 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820306+0000", 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7180000000000004e-06 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820601+0000", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029544599999999999 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.820609+0000", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3329999999999999e-06 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821192+0000", 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00058298099999999999 2024-09-17T21:46:58.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.821301+0000", 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000109826 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.828510+0000", 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072084130000000003 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2005 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 172032~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.399249689000001, 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076004749999999998, 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2005 2024-09-17T21:46:58.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832192+0000", 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-07 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832195+0000", 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.339e-06 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832201+0000", 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4629999999999998e-06 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832206+0000", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3090000000000003e-06 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839559+0000", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073530169999999999 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839641+0000", 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1210000000000003e-05 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.839792+0000", 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015160099999999999 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2006 3.18 3:186ac491:::rbd_data.1112a93a0526.0000000000000015:a [read 229376~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.399173723000001, 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083420109999999999, 2024-09-17T21:46:58.256 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2006 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832268+0000", 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4300000000000003e-07 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832272+0000", 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7290000000000002e-06 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832281+0000", 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2909999999999996e-06 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832288+0000", 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2280000000000003e-06 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840364+0000", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080765279999999995 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840441+0000", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7028999999999997e-05 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840610+0000", 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016856299999999999 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:2033 3.8 3:109bf8c5:::rbd_data.11121f8e87bb.000000000000001c:8 [read 65536~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.398997968, 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077497670000000003, 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2033 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832443+0000", 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832444+0000", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000001e-07 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832446+0000", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832453+0000", 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3180000000000003e-06 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.832457+0000", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840020+0000", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0075630999999999997 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840101+0000", 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0913000000000003e-05 2024-09-17T21:46:58.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.840193+0000", 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1823000000000005e-05 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:58.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.327920448, 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:46:58.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:58.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.326458313, 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.272 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:46:58.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:46:58.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.323847194999999, 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:46:58.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:46:58.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:46:58.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:46:58.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4125 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.323119209000001, 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098331159999999994, 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4125 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908322+0000", 2024-09-17T21:46:58.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908323+0000", 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2500000000000004e-07 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908327+0000", 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908334+0000", 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4880000000000001e-06 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908342+0000", 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8909999999999995e-06 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914269+0000", 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0059269379999999997 2024-09-17T21:46:58.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914325+0000", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6508000000000002e-05 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.914456+0000", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000130889 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918056+0000", 2024-09-17T21:46:58.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035999629999999999 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918075+0000", 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8549000000000001e-05 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918119+0000", 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4535e-05 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.918155+0000", 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5954000000000002e-05 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.259732738, 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:46:58.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:46:58.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.259004516000001, 2024-09-17T21:46:58.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:46:58.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:46:58.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.925427379999999, 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:46:58.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:46:58.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.358668878, 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:46:58.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:46:58.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:46:58.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.351146736, 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:46:58.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:46:58.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:350 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 5242880~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.384252478000001, 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0077978400000000003, 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 350 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847189+0000", 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:46:58.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847191+0000", 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0679999999999999e-06 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847203+0000", 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1554e-05 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847206+0000", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.41e-06 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853739+0000", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065326860000000002 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.853817+0000", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8072999999999996e-05 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.854987+0000", 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011696720000000001 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:351 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 16777216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.384226829999999, 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0080135269999999995, 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 351 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847215+0000", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847217+0000", 2024-09-17T21:46:58.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.668e-06 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847225+0000", 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4200000000000007e-06 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847229+0000", 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5700000000000001e-06 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855031+0000", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078027970000000002 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855115+0000", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3493999999999997e-05 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855228+0000", 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001133 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:358 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 6291456~819200] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.384176930999999, 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095067050000000007, 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:58.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 358 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847265+0000", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847267+0000", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.238e-06 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847272+0000", 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0509999999999996e-06 2024-09-17T21:46:58.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847275+0000", 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1209999999999998e-06 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855269+0000", 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079939260000000002 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.855350+0000", 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0716000000000004e-05 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856771+0000", 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014212719999999999 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4822.0:359 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:d [sparse-read 17825792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.332 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.384146903000001, 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096101729999999996, 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4822", 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 359 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847295+0000", 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847297+0000", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.148e-06 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847302+0000", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0209999999999999e-06 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.847305+0000", 2024-09-17T21:46:58.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.072e-06 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856799+0000", 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0094936829999999993 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856847+0000", 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7970000000000003e-05 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:35.856905+0000", 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7819000000000003e-05 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4822.0:730 3.d e51/21 3:b2db4994:::rbd_data.111223014882.0000000000000000:head v 51'272, mlcod=51'270)", 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.195923285999999, 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096314799999999996, 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:46:58.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035518+0000", 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.035519+0000", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.44e-07 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039382+0000", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038629329999999998 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039396+0000", 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4079e-05 2024-09-17T21:46:58.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039402+0000", 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7780000000000004e-06 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039448+0000", 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100999999999998e-05 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:46:58.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.039463+0000", 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5537e-05 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045104+0000", 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005640305 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:36.045150+0000", 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6103000000000002e-05 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:46:58.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:46:58.342 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:46:58.343 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-17T21:46:58.343 INFO:teuthology.orchestra.run.smithi165.stdout:2431 punch from 0x2f96a00 to 0x2fa5200, (0xe800 bytes) 2024-09-17T21:46:58.344 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:46:58.344 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:58.340+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:58.345 INFO:teuthology.orchestra.run.smithi165.stdout:2432 trunc from 0x97d6000 to 0x6094200 2024-09-17T21:46:58.358 INFO:teuthology.orchestra.run.smithi165.stdout:2433 read 0x45bbc00 thru 0x45c1301 (0x5702 bytes) 2024-09-17T21:46:58.358 INFO:teuthology.orchestra.run.smithi165.stdout:2434 write 0xdb1a00 thru 0xdbe408 (0xca09 bytes) 2024-09-17T21:46:58.358 INFO:teuthology.orchestra.run.smithi165.stdout:2435 write 0xf40400 thru 0xf4820a (0x7e0b bytes) 2024-09-17T21:46:58.359 INFO:teuthology.orchestra.run.smithi165.stdout:2436 write 0x4b8fc00 thru 0x4b917e7 (0x1be8 bytes) 2024-09-17T21:46:58.359 INFO:teuthology.orchestra.run.smithi165.stdout:2437 write 0x47b4c00 thru 0x47c2c0a (0xe00b bytes) 2024-09-17T21:46:58.391 INFO:teuthology.orchestra.run.smithi165.stdout:2441 write 0xd36bc00 thru 0xd37020f (0x4610 bytes) 2024-09-17T21:46:58.392 INFO:teuthology.orchestra.run.smithi165.stdout:2442 punch from 0xcfbd200 to 0xcfc5400, (0x8200 bytes) 2024-09-17T21:46:58.392 INFO:teuthology.orchestra.run.smithi165.stdout:2443 read 0xa3fb200 thru 0xa409a2e (0xe82f bytes) 2024-09-17T21:46:58.395 INFO:teuthology.orchestra.run.smithi165.stdout:2444 read 0xa72ca00 thru 0xa730bb1 (0x41b2 bytes) 2024-09-17T21:46:58.395 INFO:teuthology.orchestra.run.smithi165.stdout:2445 write 0xa378e00 thru 0xa384458 (0xb659 bytes) 2024-09-17T21:46:58.396 INFO:teuthology.orchestra.run.smithi165.stdout:2446 read 0x53e7200 thru 0x53f27c4 (0xb5c5 bytes) 2024-09-17T21:46:58.396 INFO:teuthology.orchestra.run.smithi165.stdout:2447 punch from 0x6e7a200 to 0x6e83400, (0x9200 bytes) 2024-09-17T21:46:58.396 INFO:teuthology.orchestra.run.smithi165.stdout:2448 read 0x7e0e00 thru 0x7ed299 (0xc49a bytes) 2024-09-17T21:46:58.402 INFO:teuthology.orchestra.run.smithi165.stdout:2449 trunc from 0xd370400 to 0x5c4b000 2024-09-17T21:46:58.417 INFO:teuthology.orchestra.run.smithi165.stdout:2451 read 0x303000 thru 0x30b08c (0x808d bytes) 2024-09-17T21:46:58.425 INFO:teuthology.orchestra.run.smithi165.stdout:2453 trunc from 0x5c4b000 to 0xad16c00 2024-09-17T21:46:58.433 INFO:teuthology.orchestra.run.smithi165.stdout:2458 read 0x1600200 thru 0x16036d4 (0x34d5 bytes) 2024-09-17T21:46:58.435 INFO:teuthology.orchestra.run.smithi165.stdout:2461 write 0x5262000 thru 0x5268dcb (0x6dcc bytes) 2024-09-17T21:46:58.435 INFO:teuthology.orchestra.run.smithi165.stdout:2464 read 0x33cb200 thru 0x33cb4be (0x2bf bytes) 2024-09-17T21:46:58.438 INFO:teuthology.orchestra.run.smithi165.stdout:2465 write 0x544cc00 thru 0x545be4b (0xf24c bytes) 2024-09-17T21:46:58.438 INFO:teuthology.orchestra.run.smithi165.stdout:2466 write 0x6c31000 thru 0x6c3c792 (0xb793 bytes) 2024-09-17T21:46:58.443 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:58.440+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:58.457 INFO:teuthology.orchestra.run.smithi165.stdout:2468 write 0xed3c200 thru 0xed44420 (0x8221 bytes) 2024-09-17T21:46:58.457 INFO:teuthology.orchestra.run.smithi165.stdout:2471 write 0x49b0400 thru 0x49b8a80 (0x8681 bytes) 2024-09-17T21:46:58.458 INFO:teuthology.orchestra.run.smithi165.stdout:2474 read 0x58eda00 thru 0x58fd31e (0xf91f bytes) 2024-09-17T21:46:58.459 INFO:teuthology.orchestra.run.smithi165.stdout:2475 read 0x2ee0c00 thru 0x2ee3ef3 (0x32f4 bytes) 2024-09-17T21:46:58.459 INFO:teuthology.orchestra.run.smithi165.stdout:2476 clone 20 order 24 su 65536 sc 2 2024-09-17T21:46:58.472 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:58.473 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:58.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:46:58.545 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:58.542+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:58.588 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:58.589 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:58.589 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:46:58.589 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:46:58.589 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:58.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:46:58.646 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:46:58.643+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.584344102000003, 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:46:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:46:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:46:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.923358813, 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:46:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:46:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.905787422, 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:46:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:46:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.905753729000001, 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:46:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:46:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902912426, 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:46:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902835114999998, 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:46:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902812722, 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902791601000001, 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:46:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902770857, 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:46:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902356213000001, 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.902321021999999, 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:46:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:46:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.901429559, 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:46:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.901063489999999, 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:46:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.847756272000002, 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:46:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:46:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.847720500000001, 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:46:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.843582128000001, 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:46:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:46:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.838388407, 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:46:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.445881814, 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:46:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:46:58.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:46:58.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.874582081, 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:46:58.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:46:58.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.869034718000002, 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:46:58.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:46:58.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:46:58.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:46:58.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:46:58.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:46:58.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:46:58.854 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:58.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:46:58.855 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:58.744+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:58.856 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:46:58.825+0000 7f0c2ae19640 -1 log_channel(cluster) log [ERR] : Health check failed: full ratio(s) out of order (OSD_OUT_OF_ORDER_FULL) 2024-09-17T21:46:58.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:46:58.857 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:58.853+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:58.858 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.001 2024-09-17T21:46:58.929 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone19 from 0xed44600 (overlap 0x5c4b000) to 0x0 2024-09-17T21:46:58.956 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:58.954+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:59.011 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:46:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:46:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:46:59.012 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:46:59.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:46:59.029 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:46:59.029 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:46:59.047 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:46:59.048 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:46:59.061 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:59.059+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:59.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:46:59.065 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:46:59.082 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:46:59.082 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:46:59.099 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:46:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:46:59.117 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:46:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:46:59.158 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:59.155+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:59.169 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.169 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:46:59.259 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:59.257+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.300 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:46:59.346 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-17T21:46:59.384 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:59.382+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:46:59.444 INFO:teuthology.orchestra.run.smithi165.stdout:2477 trunc from 0xed44600 to 0x17b0e00 2024-09-17T21:46:59.450 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.450 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.459 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:46:59.460 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:46:59.457+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:46:59.528 INFO:teuthology.orchestra.run.smithi165.stdout:2479 trunc from 0x17b0e00 to 0x8ec3a00 2024-09-17T21:46:59.535 INFO:teuthology.orchestra.run.smithi165.stdout:2480 read 0x259d800 thru 0x25a239f (0x4ba0 bytes) 2024-09-17T21:46:59.536 INFO:teuthology.orchestra.run.smithi165.stdout:2481 punch from 0x6cd2000 to 0x6cd7600, (0x5600 bytes) 2024-09-17T21:46:59.537 INFO:teuthology.orchestra.run.smithi165.stdout:2482 read 0x5ffe800 thru 0x60080f1 (0x98f2 bytes) 2024-09-17T21:46:59.549 INFO:teuthology.orchestra.run.smithi165.stdout:2483 write 0x9116600 thru 0x911aee3 (0x48e4 bytes) 2024-09-17T21:46:59.561 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:46:59.558+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:46:59.565 INFO:teuthology.orchestra.run.smithi165.stdout:2484 write 0xc91b000 thru 0xc9257ec (0xa7ed bytes) 2024-09-17T21:46:59.566 INFO:teuthology.orchestra.run.smithi165.stdout:2485 write 0x492dc00 thru 0x49304e1 (0x28e2 bytes) 2024-09-17T21:46:59.582 INFO:teuthology.orchestra.run.smithi165.stdout:2487 write 0xe404400 thru 0xe40c394 (0x7f95 bytes) 2024-09-17T21:46:59.584 INFO:teuthology.orchestra.run.smithi165.stdout:2488 write 0x545ea00 thru 0x5469843 (0xae44 bytes) 2024-09-17T21:46:59.584 INFO:teuthology.orchestra.run.smithi165.stdout:2493 read 0x7170a00 thru 0x717277f (0x1d80 bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2494 read 0x9619400 thru 0x961a20a (0xe0b bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2497 write 0x1321e00 thru 0x1328dba (0x6fbb bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2499 read 0x711d400 thru 0x7129119 (0xbd1a bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2500 read 0x1238a00 thru 0x1243943 (0xaf44 bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2501 write 0x289a000 thru 0x28a16f0 (0x76f1 bytes) 2024-09-17T21:46:59.588 INFO:teuthology.orchestra.run.smithi165.stdout:2505 write 0xda76e00 thru 0xda83928 (0xcb29 bytes) 2024-09-17T21:46:59.589 INFO:teuthology.orchestra.run.smithi165.stdout:2506 punch from 0x7f79600 to 0x7f81a00, (0x8400 bytes) 2024-09-17T21:46:59.603 INFO:teuthology.orchestra.run.smithi165.stdout:2507 trunc from 0xe40c400 to 0x27b600 2024-09-17T21:46:59.615 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.616 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.625 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:46:59.643 INFO:teuthology.orchestra.run.smithi165.stdout:2508 write 0xc033e00 thru 0xc03b0db (0x72dc bytes) 2024-09-17T21:46:59.647 INFO:teuthology.orchestra.run.smithi165.stdout:2509 punch from 0x70b7c00 to 0x70c4c00, (0xd000 bytes) 2024-09-17T21:46:59.648 INFO:teuthology.orchestra.run.smithi165.stdout:2510 read 0x9da4800 thru 0x9da4bfc (0x3fd bytes) 2024-09-17T21:46:59.648 INFO:teuthology.orchestra.run.smithi165.stdout:2511 write 0x5a52000 thru 0x5a5c31c (0xa31d bytes) 2024-09-17T21:46:59.662 INFO:teuthology.orchestra.run.smithi165.stdout:2517 write 0xde3d200 thru 0xde40451 (0x3252 bytes) 2024-09-17T21:46:59.667 INFO:teuthology.orchestra.run.smithi165.stdout:2518 read 0x946fc00 thru 0x9478b7d (0x8f7e bytes) 2024-09-17T21:46:59.668 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:46:59.666+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:46:59.669 INFO:teuthology.orchestra.run.smithi165.stdout:2520 trunc from 0xde40600 to 0x812c200 2024-09-17T21:46:59.684 INFO:teuthology.orchestra.run.smithi165.stdout:2521 read 0x973c00 thru 0x974f06 (0x1307 bytes) 2024-09-17T21:46:59.686 INFO:teuthology.orchestra.run.smithi165.stdout:2522 punch from 0x55f7000 to 0x55fb400, (0x4400 bytes) 2024-09-17T21:46:59.690 INFO:teuthology.orchestra.run.smithi165.stdout:2523 read 0x7fc4600 thru 0x7fc94ff (0x4f00 bytes) 2024-09-17T21:46:59.691 INFO:teuthology.orchestra.run.smithi165.stdout:2524 trunc from 0x812c200 to 0x881f000 2024-09-17T21:46:59.715 INFO:teuthology.orchestra.run.smithi165.stdout:2525 write 0xdfd9800 thru 0xdfda00c (0x80d bytes) 2024-09-17T21:46:59.716 INFO:teuthology.orchestra.run.smithi165.stdout:2526 write 0xc981e00 thru 0xc991b5b (0xfd5c bytes) 2024-09-17T21:46:59.716 INFO:teuthology.orchestra.run.smithi165.stdout:2527 punch from 0xd74a400 to 0xd74c000, (0x1c00 bytes) 2024-09-17T21:46:59.717 INFO:teuthology.orchestra.run.smithi165.stdout:2530 read 0xaedc00 thru 0xaf82a9 (0xa6aa bytes) 2024-09-17T21:46:59.717 INFO:teuthology.orchestra.run.smithi165.stdout:2532 punch from 0x27ee400 to 0x27f5800, (0x7400 bytes) 2024-09-17T21:46:59.718 INFO:teuthology.orchestra.run.smithi165.stdout:2533 read 0xd7bb600 thru 0xd7c6fbc (0xb9bd bytes) 2024-09-17T21:46:59.718 INFO:teuthology.orchestra.run.smithi165.stdout:2534 write 0x6da7600 thru 0x6db027c (0x8c7d bytes) 2024-09-17T21:46:59.718 INFO:teuthology.orchestra.run.smithi165.stdout:2535 read 0x4cd4e00 thru 0x4cdbcab (0x6eac bytes) 2024-09-17T21:46:59.722 INFO:teuthology.orchestra.run.smithi165.stdout:2536 trunc from 0xdfda200 to 0xacc7200 2024-09-17T21:46:59.736 INFO:teuthology.orchestra.run.smithi165.stdout:2537 read 0x6739e00 thru 0x673ab79 (0xd7a bytes) 2024-09-17T21:46:59.739 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.740 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.740 INFO:teuthology.orchestra.run.smithi165.stdout:2539 trunc from 0xacc7200 to 0xcdb0600 2024-09-17T21:46:59.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:46:59.750 INFO:teuthology.orchestra.run.smithi165.stdout:2542 clone 21 order 20 su 65536 sc 9 2024-09-17T21:46:59.791 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:59.788+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:59.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:46:59.858 INFO:tasks.thrashosds.thrasher:Setting full ratio back to .95 2024-09-17T21:46:59.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set-full-ratio .95 2024-09-17T21:46:59.864 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:46:59.864 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:46:59.864 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:46:59.862+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:46:59.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:46:59.949 INFO:teuthology.orchestra.run.smithi165.stdout:flattening image image_client.0-clone20 2024-09-17T21:46:59.965 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:46:59.963+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:00.042 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.042 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:00.066 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:00.064+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:00.163 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.163 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.167 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:00.165+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:00.172 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:00.269 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:00.266+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:00.280 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.280 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:00.312 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-17T21:47:00.369 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:00.367+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:00.371 INFO:teuthology.orchestra.run.smithi165.stdout:2546 read 0x185000 thru 0x18f480 (0xa481 bytes) 2024-09-17T21:47:00.375 INFO:teuthology.orchestra.run.smithi165.stdout:2548 write 0xa86e00 thru 0xa92141 (0xb342 bytes) 2024-09-17T21:47:00.397 INFO:teuthology.orchestra.run.smithi165.stdout:2551 write 0xdec1e00 thru 0xdecd641 (0xb842 bytes) 2024-09-17T21:47:00.398 INFO:teuthology.orchestra.run.smithi165.stdout:2553 punch from 0xb31e600 to 0xb329200, (0xac00 bytes) 2024-09-17T21:47:00.398 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.398 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.399 INFO:teuthology.orchestra.run.smithi165.stdout:2556 write 0xe2f600 thru 0xe37486 (0x7e87 bytes) 2024-09-17T21:47:00.400 INFO:teuthology.orchestra.run.smithi165.stdout:2557 read 0x3b8d600 thru 0x3b963c0 (0x8dc1 bytes) 2024-09-17T21:47:00.400 INFO:teuthology.orchestra.run.smithi165.stdout:2558 write 0x1f49200 thru 0x1f549e2 (0xb7e3 bytes) 2024-09-17T21:47:00.401 INFO:teuthology.orchestra.run.smithi165.stdout:2559 write 0x2385600 thru 0x238738e (0x1d8f bytes) 2024-09-17T21:47:00.401 INFO:teuthology.orchestra.run.smithi165.stdout:2560 write 0x1c57400 thru 0x1c59852 (0x2453 bytes) 2024-09-17T21:47:00.407 INFO:teuthology.orchestra.run.smithi165.stdout:2562 trunc from 0xdecd800 to 0x98d0c00 2024-09-17T21:47:00.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:00.439 INFO:teuthology.orchestra.run.smithi165.stdout:2567 read 0x9499800 thru 0x94a849f (0xeca0 bytes) 2024-09-17T21:47:00.447 INFO:teuthology.orchestra.run.smithi165.stdout:2568 trunc from 0x98d0c00 to 0x355b600 2024-09-17T21:47:00.470 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:00.467+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:00.481 INFO:teuthology.orchestra.run.smithi165.stdout:2569 flatten 2024-09-17T21:47:00.532 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.532 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.537 INFO:teuthology.orchestra.run.smithi165.stdout:2570 read 0x751200 thru 0x75e377 (0xd178 bytes) 2024-09-17T21:47:00.538 INFO:teuthology.orchestra.run.smithi165.stdout:2573 read 0x30f7c00 thru 0x30fef0e (0x730f bytes) 2024-09-17T21:47:00.539 INFO:teuthology.orchestra.run.smithi165.stdout:2576 read 0x1f52e00 thru 0x1f6164e (0xe84f bytes) 2024-09-17T21:47:00.540 INFO:teuthology.orchestra.run.smithi165.stdout:2577 read 0x1ba9c00 thru 0x1bb8e0f (0xf210 bytes) 2024-09-17T21:47:00.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:00.553 INFO:teuthology.orchestra.run.smithi165.stdout:2578 write 0x74a2200 thru 0x74a349b (0x129c bytes) 2024-09-17T21:47:00.555 INFO:teuthology.orchestra.run.smithi165.stdout:2579 read 0x6f96e00 thru 0x6f9be67 (0x5068 bytes) 2024-09-17T21:47:00.571 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:00.568+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:00.574 INFO:teuthology.orchestra.run.smithi165.stdout:2581 write 0xd563a00 thru 0xd56cff8 (0x95f9 bytes) 2024-09-17T21:47:00.575 INFO:teuthology.orchestra.run.smithi165.stdout:2582 read 0x995d000 thru 0x996a775 (0xd776 bytes) 2024-09-17T21:47:00.589 INFO:teuthology.orchestra.run.smithi165.stdout:2583 trunc from 0xd56d000 to 0x2654200 2024-09-17T21:47:00.629 INFO:teuthology.orchestra.run.smithi165.stdout:2584 trunc from 0x2654200 to 0xdd59000 2024-09-17T21:47:00.645 INFO:teuthology.orchestra.run.smithi165.stdout:2586 write 0x832800 thru 0x83cf6e (0xa76f bytes) 2024-09-17T21:47:00.646 INFO:teuthology.orchestra.run.smithi165.stdout:2587 read 0xaf70800 thru 0xaf773bb (0x6bbc bytes) 2024-09-17T21:47:00.646 INFO:teuthology.orchestra.run.smithi165.stdout:2588 write 0xbdd6000 thru 0xbddcdef (0x6df0 bytes) 2024-09-17T21:47:00.646 INFO:teuthology.orchestra.run.smithi165.stdout:2589 write 0x1b02c00 thru 0x1b0d5f0 (0xa9f1 bytes) 2024-09-17T21:47:00.653 INFO:teuthology.orchestra.run.smithi165.stdout:2592 trunc from 0xdd59000 to 0x80ba400 2024-09-17T21:47:00.670 INFO:teuthology.orchestra.run.smithi165.stdout:2593 punch from 0x1f5f600 to 0x1f69800, (0xa200 bytes) 2024-09-17T21:47:00.671 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:00.668+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:00.672 INFO:teuthology.orchestra.run.smithi165.stdout:2596 write 0x4730800 thru 0x4735a1a (0x521b bytes) 2024-09-17T21:47:00.675 INFO:teuthology.orchestra.run.smithi165.stdout:2598 write 0x57fd600 thru 0x580888b (0xb28c bytes) 2024-09-17T21:47:00.676 INFO:teuthology.orchestra.run.smithi165.stdout:2599 read 0x2576200 thru 0x257edc9 (0x8bca bytes) 2024-09-17T21:47:00.676 INFO:teuthology.orchestra.run.smithi165.stdout:2600 trunc from 0x80ba400 to 0x977bc00 2024-09-17T21:47:00.687 INFO:teuthology.orchestra.run.smithi165.stdout:2604 punch from 0x1f61800 to 0x1f68e00, (0x7600 bytes) 2024-09-17T21:47:00.688 INFO:teuthology.orchestra.run.smithi165.stdout:2608 read 0x19ff600 thru 0x1a0f2ec (0xfced bytes) 2024-09-17T21:47:00.688 INFO:teuthology.orchestra.run.smithi165.stdout:2609 write 0x530a00 thru 0x539067 (0x8668 bytes) 2024-09-17T21:47:00.689 INFO:teuthology.orchestra.run.smithi165.stdout:2610 punch from 0x46ab600 to 0x46b8a00, (0xd400 bytes) 2024-09-17T21:47:00.689 INFO:teuthology.orchestra.run.smithi165.stdout:2611 read 0x9163e00 thru 0x916852f (0x4730 bytes) 2024-09-17T21:47:00.689 INFO:teuthology.orchestra.run.smithi165.stdout:2612 read 0x43df400 thru 0x43ee846 (0xf447 bytes) 2024-09-17T21:47:00.691 INFO:teuthology.orchestra.run.smithi165.stdout:2614 write 0x4bad200 thru 0x4baf2cd (0x20ce bytes) 2024-09-17T21:47:00.691 INFO:teuthology.orchestra.run.smithi165.stdout:2618 punch from 0x8c1a600 to 0x8c25e00, (0xb800 bytes) 2024-09-17T21:47:00.691 INFO:teuthology.orchestra.run.smithi165.stdout:2621 read 0x580e400 thru 0x581bc0a (0xd80b bytes) 2024-09-17T21:47:00.698 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.698 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.703 INFO:teuthology.orchestra.run.smithi165.stdout:2622 write 0xa42b200 thru 0xa42cd5c (0x1b5d bytes) 2024-09-17T21:47:00.705 INFO:teuthology.orchestra.run.smithi165.stdout:2623 read 0x996c00 thru 0x99bd9a (0x519b bytes) 2024-09-17T21:47:00.705 INFO:teuthology.orchestra.run.smithi165.stdout:2624 read 0x920c400 thru 0x92100d5 (0x3cd6 bytes) 2024-09-17T21:47:00.705 INFO:teuthology.orchestra.run.smithi165.stdout:2625 write 0x73a2400 thru 0x73ae20c (0xbe0d bytes) 2024-09-17T21:47:00.705 INFO:teuthology.orchestra.run.smithi165.stdout:2626 punch from 0x245c200 to 0x246b400, (0xf200 bytes) 2024-09-17T21:47:00.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:00.719 INFO:teuthology.orchestra.run.smithi165.stdout:2628 write 0xc2aa200 thru 0xc2acd20 (0x2b21 bytes) 2024-09-17T21:47:00.720 INFO:teuthology.orchestra.run.smithi165.stdout:2631 write 0xba1a00 thru 0xbaeb52 (0xd153 bytes) 2024-09-17T21:47:00.720 INFO:teuthology.orchestra.run.smithi165.stdout:2633 read 0x4a1da00 thru 0x4a23001 (0x5602 bytes) 2024-09-17T21:47:00.720 INFO:teuthology.orchestra.run.smithi165.stdout:2634 punch from 0x7499800 to 0x74a1e00, (0x8600 bytes) 2024-09-17T21:47:00.721 INFO:teuthology.orchestra.run.smithi165.stdout:2636 write 0x4131600 thru 0x413cc41 (0xb642 bytes) 2024-09-17T21:47:00.721 INFO:teuthology.orchestra.run.smithi165.stdout:2638 punch from 0x6d24000 to 0x6d2c800, (0x8800 bytes) 2024-09-17T21:47:00.721 INFO:teuthology.orchestra.run.smithi165.stdout:2639 write 0x72f6400 thru 0x73031ba (0xcdbb bytes) 2024-09-17T21:47:00.722 INFO:teuthology.orchestra.run.smithi165.stdout:2641 read 0x9aec600 thru 0x9afac36 (0xe637 bytes) 2024-09-17T21:47:00.722 INFO:teuthology.orchestra.run.smithi165.stdout:2643 read 0x9013000 thru 0x9021faf (0xefb0 bytes) 2024-09-17T21:47:00.728 INFO:teuthology.orchestra.run.smithi165.stdout:2644 trunc from 0xc2ace00 to 0x6ec7200 2024-09-17T21:47:00.745 INFO:teuthology.orchestra.run.smithi165.stdout:2645 punch from 0x39b6c00 to 0x39c6400, (0xf800 bytes) 2024-09-17T21:47:00.746 INFO:teuthology.orchestra.run.smithi165.stdout:2646 punch from 0x2751000 to 0x275ce00, (0xbe00 bytes) 2024-09-17T21:47:00.759 INFO:teuthology.orchestra.run.smithi165.stdout:2648 write 0x7747000 thru 0x774f0db (0x80dc bytes) 2024-09-17T21:47:00.760 INFO:teuthology.orchestra.run.smithi165.stdout:2649 punch from 0x7718200 to 0x7718800, (0x600 bytes) 2024-09-17T21:47:00.760 INFO:teuthology.orchestra.run.smithi165.stdout:2655 punch from 0x421600 to 0x430000, (0xea00 bytes) 2024-09-17T21:47:00.775 INFO:teuthology.orchestra.run.smithi165.stdout:2656 write 0xc005400 thru 0xc010fc8 (0xbbc9 bytes) 2024-09-17T21:47:00.776 INFO:teuthology.orchestra.run.smithi165.stdout:2658 write 0x9a1000 thru 0x9a20ae (0x10af bytes) 2024-09-17T21:47:00.776 INFO:teuthology.orchestra.run.smithi165.stdout:2659 punch from 0x547d800 to 0x548bc00, (0xe400 bytes) 2024-09-17T21:47:00.777 INFO:teuthology.orchestra.run.smithi165.stdout:2660 punch from 0xa618c00 to 0xa624600, (0xba00 bytes) 2024-09-17T21:47:00.777 INFO:teuthology.orchestra.run.smithi165.stdout:2663 punch from 0x60d000 to 0x610000, (0x3000 bytes) 2024-09-17T21:47:00.791 INFO:teuthology.orchestra.run.smithi165.stdout:2666 trunc from 0xc011000 to 0x2ef400 2024-09-17T21:47:00.792 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:00.789+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:00.834 INFO:teuthology.orchestra.run.smithi165.stdout:2667 trunc from 0x2ef400 to 0x9078c00 2024-09-17T21:47:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:osd set-full-ratio 0.95 2024-09-17T21:47:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:47:00.857 INFO:teuthology.orchestra.run.smithi165.stdout:2668 trunc from 0x9078c00 to 0x77b400 2024-09-17T21:47:00.872 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:00.872 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:00.873 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:00.871+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:00.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:00.895 INFO:teuthology.orchestra.run.smithi165.stdout:2669 write 0x5e90000 thru 0x5e9b5b0 (0xb5b1 bytes) 2024-09-17T21:47:00.904 INFO:teuthology.orchestra.run.smithi165.stdout:2671 trunc from 0x5e9b600 to 0xcab2e00 2024-09-17T21:47:00.921 INFO:teuthology.orchestra.run.smithi165.stdout:2672 read 0x5a000 thru 0x5fdab (0x5dac bytes) 2024-09-17T21:47:00.922 INFO:teuthology.orchestra.run.smithi165.stdout:2673 write 0x88b8c00 thru 0x88c4f4e (0xc34f bytes) 2024-09-17T21:47:00.922 INFO:teuthology.orchestra.run.smithi165.stdout:2675 read 0x2ae6400 thru 0x2af5c66 (0xf867 bytes) 2024-09-17T21:47:00.922 INFO:teuthology.orchestra.run.smithi165.stdout:2676 read 0x71a1400 thru 0x71a65bd (0x51be bytes) 2024-09-17T21:47:00.934 INFO:teuthology.orchestra.run.smithi165.stdout:2678 trunc from 0xcab2e00 to 0x1df4200 2024-09-17T21:47:00.959 INFO:teuthology.orchestra.run.smithi165.stdout:2680 punch from 0x12ade00 to 0x12b0400, (0x2600 bytes) 2024-09-17T21:47:00.974 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:00.972+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:00.976 INFO:teuthology.orchestra.run.smithi165.stdout:2681 write 0x1e12400 thru 0x1e1efdd (0xcbde bytes) 2024-09-17T21:47:00.976 INFO:teuthology.orchestra.run.smithi165.stdout:2682 read 0x181ae00 thru 0x182a8e9 (0xfaea bytes) 2024-09-17T21:47:00.988 INFO:teuthology.orchestra.run.smithi165.stdout:2688 write 0x2534400 thru 0x2536b08 (0x2709 bytes) 2024-09-17T21:47:00.997 INFO:teuthology.orchestra.run.smithi165.stdout:2690 trunc from 0x2536c00 to 0x870da00 2024-09-17T21:47:01.010 INFO:teuthology.orchestra.run.smithi165.stdout:2692 trunc from 0x870da00 to 0x78c8a00 2024-09-17T21:47:01.025 INFO:teuthology.orchestra.run.smithi165.stdout:2693 punch from 0x279e000 to 0x27a5400, (0x7400 bytes) 2024-09-17T21:47:01.032 INFO:teuthology.orchestra.run.smithi165.stdout:2694 read 0x3ce3c00 thru 0x3cea50d (0x690e bytes) 2024-09-17T21:47:01.037 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:01.037 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:01.046 INFO:teuthology.orchestra.run.smithi165.stdout:2695 write 0xbb3c400 thru 0xbb49e6d (0xda6e bytes) 2024-09-17T21:47:01.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:01.048 INFO:teuthology.orchestra.run.smithi165.stdout:2696 read 0x8539a00 thru 0x85431f0 (0x97f1 bytes) 2024-09-17T21:47:01.060 INFO:teuthology.orchestra.run.smithi165.stdout:2697 write 0xbf54a00 thru 0xbf61d80 (0xd381 bytes) 2024-09-17T21:47:01.061 INFO:teuthology.orchestra.run.smithi165.stdout:2698 punch from 0x9e83c00 to 0x9e89a00, (0x5e00 bytes) 2024-09-17T21:47:01.062 INFO:teuthology.orchestra.run.smithi165.stdout:2699 punch from 0x3014800 to 0x301e200, (0x9a00 bytes) 2024-09-17T21:47:01.062 INFO:teuthology.orchestra.run.smithi165.stdout:2700 write 0x6951200 thru 0x695e584 (0xd385 bytes) 2024-09-17T21:47:01.062 INFO:teuthology.orchestra.run.smithi165.stdout:2701 read 0x3db9800 thru 0x3dbe5ae (0x4daf bytes) 2024-09-17T21:47:01.062 INFO:teuthology.orchestra.run.smithi165.stdout:2703 punch from 0xa173c00 to 0xa177c00, (0x4000 bytes) 2024-09-17T21:47:01.063 INFO:teuthology.orchestra.run.smithi165.stdout:2706 read 0x5840a00 thru 0x584945b (0x8a5c bytes) 2024-09-17T21:47:01.063 INFO:teuthology.orchestra.run.smithi165.stdout:2709 read 0x405ca00 thru 0x4063e9d (0x749e bytes) 2024-09-17T21:47:01.063 INFO:teuthology.orchestra.run.smithi165.stdout:2710 trunc from 0xbf61e00 to 0xc63ae00 2024-09-17T21:47:01.077 INFO:teuthology.orchestra.run.smithi165.stdout:2711 write 0x6f5ac00 thru 0x6f5bdd6 (0x11d7 bytes) 2024-09-17T21:47:01.084 INFO:teuthology.orchestra.run.smithi165.stdout:2712 read 0x68b3c00 thru 0x68c27c4 (0xebc5 bytes) 2024-09-17T21:47:01.084 INFO:teuthology.orchestra.run.smithi165.stdout:2714 write 0x4054e00 thru 0x4057653 (0x2854 bytes) 2024-09-17T21:47:01.084 INFO:teuthology.orchestra.run.smithi165.stdout:2716 read 0x3360800 thru 0x33667f2 (0x5ff3 bytes) 2024-09-17T21:47:01.084 INFO:teuthology.orchestra.run.smithi165.stdout:2718 punch from 0x3621800 to 0x3630200, (0xea00 bytes) 2024-09-17T21:47:01.085 INFO:teuthology.orchestra.run.smithi165.stdout:2719 punch from 0xbabc200 to 0xbac1800, (0x5600 bytes) 2024-09-17T21:47:01.085 INFO:teuthology.orchestra.run.smithi165.stdout:2721 punch from 0x3fd800 to 0x409400, (0xbc00 bytes) 2024-09-17T21:47:01.085 INFO:teuthology.orchestra.run.smithi165.stdout:2722 write 0x28f000 thru 0x29a1f6 (0xb1f7 bytes) 2024-09-17T21:47:01.086 INFO:teuthology.orchestra.run.smithi165.stdout:2724 write 0x2320800 thru 0x23270f8 (0x68f9 bytes) 2024-09-17T21:47:01.086 INFO:teuthology.orchestra.run.smithi165.stdout:2725 write 0x5be8000 thru 0x5bebe51 (0x3e52 bytes) 2024-09-17T21:47:01.086 INFO:teuthology.orchestra.run.smithi165.stdout:2727 punch from 0xa2b0e00 to 0xa2b6000, (0x5200 bytes) 2024-09-17T21:47:01.087 INFO:teuthology.orchestra.run.smithi165.stdout:2729 write 0x83e4000 thru 0x83ed53f (0x9540 bytes) 2024-09-17T21:47:01.087 INFO:teuthology.orchestra.run.smithi165.stdout:2730 punch from 0x7a14c00 to 0x7a17200, (0x2600 bytes) 2024-09-17T21:47:01.087 INFO:teuthology.orchestra.run.smithi165.stdout:2734 write 0xc239a00 thru 0xc23c380 (0x2981 bytes) 2024-09-17T21:47:01.087 INFO:teuthology.orchestra.run.smithi165.stdout:2736 read 0x333ec00 thru 0x3343106 (0x4507 bytes) 2024-09-17T21:47:01.091 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:01.089+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:01.099 INFO:teuthology.orchestra.run.smithi165.stdout:2737 trunc from 0xc63ae00 to 0x28d4800 2024-09-17T21:47:01.123 INFO:teuthology.orchestra.run.smithi165.stdout:2738 trunc from 0x28d4800 to 0x5a8200 2024-09-17T21:47:01.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:01.146 INFO:teuthology.orchestra.run.smithi165.stdout:2739 trunc from 0x5a8200 to 0x4346e00 2024-09-17T21:47:01.157 INFO:teuthology.orchestra.run.smithi165.stdout:2740 write 0x328cc00 thru 0x3299086 (0xc487 bytes) 2024-09-17T21:47:01.173 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:01.174 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:01.175 INFO:teuthology.orchestra.run.smithi165.stdout:2742 write 0x4c3ee00 thru 0x4c44d94 (0x5f95 bytes) 2024-09-17T21:47:01.176 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:01.172+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:01.181 INFO:teuthology.orchestra.run.smithi165.stdout:2743 read 0x3673600 thru 0x367a0a3 (0x6aa4 bytes) 2024-09-17T21:47:01.182 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:01.185 INFO:teuthology.orchestra.run.smithi165.stdout:2744 trunc from 0x4c44e00 to 0x2216a00 2024-09-17T21:47:01.202 INFO:teuthology.orchestra.run.smithi165.stdout:2747 read 0x6d1a00 thru 0x6d981e (0x7e1f bytes) 2024-09-17T21:47:01.209 INFO:teuthology.orchestra.run.smithi165.stdout:2750 read 0x1c76000 thru 0x1c78668 (0x2669 bytes) 2024-09-17T21:47:01.209 INFO:teuthology.orchestra.run.smithi165.stdout:2752 read 0x721c00 thru 0x727607 (0x5a08 bytes) 2024-09-17T21:47:01.222 INFO:teuthology.orchestra.run.smithi165.stdout:2754 write 0x4ae7600 thru 0x4aecda4 (0x57a5 bytes) 2024-09-17T21:47:01.242 INFO:teuthology.orchestra.run.smithi165.stdout:2755 write 0xc378a00 thru 0xc37d290 (0x4891 bytes) 2024-09-17T21:47:01.246 INFO:teuthology.orchestra.run.smithi165.stdout:2756 trunc from 0xc37d400 to 0xe1ec800 2024-09-17T21:47:01.260 INFO:teuthology.orchestra.run.smithi165.stdout:2757 write 0x1605000 thru 0x160f1f0 (0xa1f1 bytes) 2024-09-17T21:47:01.261 INFO:teuthology.orchestra.run.smithi165.stdout:2761 clone 22 order 23 su 2097152 sc 7 2024-09-17T21:47:01.262 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:01.263 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:01.277 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:01.274+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:01.280 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:01.280 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:01.296 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:01.314 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:01.332 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:01.332 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:01.337 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:01.337 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:01.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:01.349 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:01.349 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:01.400 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:01.397+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 74.293056174, 2024-09-17T21:47:01.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:47:01.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:47:01.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.462 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.632070885000001, 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:01.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:47:01.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.614499494, 2024-09-17T21:47:01.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:47:01.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:47:01.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.614465801000001, 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611624498000001, 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.476 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:47:01.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611547186999999, 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.481 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611524794000001, 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:01.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:47:01.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:47:01.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611503673000001, 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:47:01.489 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:47:01.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611482929000001, 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:47:01.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611068285000002, 2024-09-17T21:47:01.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:47:01.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:47:01.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.611033094, 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:47:01.504 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:47:01.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:47:01.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:01.509 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.610141631000001, 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:01.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:47:01.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:47:01.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.516 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.609775561999999, 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.517 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:47:01.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:47:01.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.556468343999999, 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:47:01.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:47:01.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.556432571999999, 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.526 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:47:01.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.552294199999999, 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:47:01.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:01.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:01.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.547100479000001, 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:01.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:47:01.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:47:01.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:47:01.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.154593886000001, 2024-09-17T21:47:01.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:47:01.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:47:01.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:47:01.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:47:01.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.583294153000001, 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:47:01.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.577746789999999, 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:47:01.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:47:01.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:47:01.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.561 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:01.562 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:01.560+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:01.563 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:01.663 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:01.660+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:01.670 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:01.670 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:01.670 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:01.678 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:01.764 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:01.761+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:01.825 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:01.825 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:01.833 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:01.874 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:01.872+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:01.966 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:01.963+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.028708872000003, 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:01.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.026683820000002, 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:01.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:01.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:01.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:01.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.012697764000002, 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:01.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:01.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.010736557000001, 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:01.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:01.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:02.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.008703199999999, 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:02.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:02.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.007 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.006725281999998, 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:02.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.006702488000002, 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.013 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:02.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:02.017 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.000793424999998, 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:02.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:02.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:02.022 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.998762366000001, 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:02.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.998735791999998, 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:02.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:02.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.998125999000003, 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:02.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:02.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.998108412999997, 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:02.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:02.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:02.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.998093455999999, 2024-09-17T21:47:02.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:02.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:02.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.282411967999998, 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:02.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.282381346999998, 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.059 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:02.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:02.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.282356729, 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.066 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:02.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.282334782, 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:02.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.078 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.282313311999999, 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.079 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:02.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:02.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:02.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.085 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.240896509000002, 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:02.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:02.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:02.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.239852049, 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:02.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:02.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.097 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:02.098 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:02.095+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:02.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:02.199 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:02.197+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:02.207 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:02.207 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:02.207 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:02.208 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:02.216 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:02.268 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-17T21:47:02.300 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:02.298+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:02.364 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:02.365 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:02.365 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:02.374 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:02.417 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:02.415+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:02.502 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:02.499+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.514 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539916943000001, 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:02.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:02.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539894449999998, 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:02.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:02.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539870190000002, 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:02.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539844307000003, 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:02.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:02.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539819305999998, 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:02.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539779506999999, 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:02.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539763762, 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:02.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:02.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:02.546 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539752227000001, 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:02.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:02.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.539741927000001, 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:02.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:02.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537770684000002, 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537757536000001, 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:02.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537746929999997, 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537736592000002, 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537726585000001, 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537716633999999, 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.537701325999997, 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.535725438999997, 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.535714483, 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.535701994, 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.535691530999998, 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:02.606 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:02.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:02.608 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:02.603+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:02.708 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:02.705+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:02.716 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:02.717 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:02.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:02.809 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:02.806+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:02.832 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:02.832 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:02.833 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:02.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:02.849 INFO:teuthology.orchestra.run.smithi165.stdout:2762 write 0xc8b4a00 thru 0xc8c2929 (0xdf2a bytes) 2024-09-17T21:47:02.850 INFO:teuthology.orchestra.run.smithi165.stdout:2765 write 0x39c5800 thru 0x39d1702 (0xbf03 bytes) 2024-09-17T21:47:02.850 INFO:teuthology.orchestra.run.smithi165.stdout:2766 punch from 0x61adc00 to 0x61bb600, (0xda00 bytes) 2024-09-17T21:47:02.862 INFO:teuthology.orchestra.run.smithi165.stdout:2767 trunc from 0xe1ec800 to 0xdda4200 2024-09-17T21:47:02.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:47:02.877 INFO:teuthology.orchestra.run.smithi165.stdout:2768 write 0x63e2200 thru 0x63e28c4 (0x6c5 bytes) 2024-09-17T21:47:02.883 INFO:teuthology.orchestra.run.smithi165.stdout:2770 read 0x3100a00 thru 0x3102250 (0x1851 bytes) 2024-09-17T21:47:02.883 INFO:teuthology.orchestra.run.smithi165.stdout:2771 write 0x7a3d600 thru 0x7a416a0 (0x40a1 bytes) 2024-09-17T21:47:02.883 INFO:teuthology.orchestra.run.smithi165.stdout:2775 write 0x92d0a00 thru 0x92d2701 (0x1d02 bytes) 2024-09-17T21:47:02.883 INFO:teuthology.orchestra.run.smithi165.stdout:2777 trunc from 0xdda4200 to 0xd7a3600 2024-09-17T21:47:02.911 INFO:teuthology.orchestra.run.smithi165.stdout:2780 write 0xddc6800 thru 0xddc7dd0 (0x15d1 bytes) 2024-09-17T21:47:02.916 INFO:teuthology.orchestra.run.smithi165.stdout:2782 trunc from 0xddc7e00 to 0xe587600 2024-09-17T21:47:02.926 INFO:teuthology.orchestra.run.smithi165.stdout:2783 read 0x31ad600 thru 0x31b55f6 (0x7ff7 bytes) 2024-09-17T21:47:02.928 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:02.925+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:02.929 INFO:teuthology.orchestra.run.smithi165.stdout:2784 punch from 0xd336600 to 0xd341c00, (0xb600 bytes) 2024-09-17T21:47:02.929 INFO:teuthology.orchestra.run.smithi165.stdout:2785 punch from 0xdef2400 to 0xdeffc00, (0xd800 bytes) 2024-09-17T21:47:02.944 INFO:teuthology.orchestra.run.smithi165.stdout:2786 write 0xeb8be00 thru 0xeb9b58b (0xf78c bytes) 2024-09-17T21:47:02.953 INFO:teuthology.orchestra.run.smithi165.stdout:2792 trunc from 0xeb9b600 to 0x6f0ec00 2024-09-17T21:47:02.975 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:02.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.688727243999999, 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:47:02.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:47:02.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:47:02.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.666800440000003, 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:02.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:02.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:47:02.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.315412404, 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:47:02.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:47:02.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:02.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:47:02.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:47:02.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.810473549999998, 2024-09-17T21:47:02.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:47:02.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:02.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:47:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.810463939000002, 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.810453107999997, 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:03.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.810439516999999, 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.810429167000002, 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:47:03.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.800990095000003, 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:47:03.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.800850885999999, 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.023 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.800833896, 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:47:03.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.032 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.800812897999997, 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:47:03.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:513 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24850432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.795966743999998, 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032416839999999999, 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 513 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175555+0000", 2024-09-17T21:47:03.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.11e-07 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175556+0000", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.029e-06 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175561+0000", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5149999999999997e-06 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175564+0000", 2024-09-17T21:47:03.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.694e-06 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178689+0000", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00312489 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178735+0000", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6391999999999999e-05 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178796+0000", 2024-09-17T21:47:03.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0952999999999997e-05 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.795954334999998, 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:03.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.795944003000002, 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:47:03.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.288700106999997, 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:03.051 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:03.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:03.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:03.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:03.057 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.256425131999997, 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:03.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:03.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:03.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.256186364000001, 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:03.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:03.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:03.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.111476748999999, 2024-09-17T21:47:03.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:03.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5145.0:362 3.6 3:64a48ca3:::rbd_data.1112b4c08b07.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 7016448~57344,write 7602176~36864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.5984145959999996, 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065350929999999996, 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:03.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5145", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/187823783", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:03.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373108+0000", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374613+0000", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015053499999999999 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374626+0000", 2024-09-17T21:47:03.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2616000000000001e-05 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374634+0000", 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1829999999999994e-06 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374681+0000", 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6828e-05 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:03.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374778+0000", 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7036999999999999e-05 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.375069+0000", 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029081000000000002 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.378468+0000", 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033987090000000002 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379563+0000", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001095155 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379603+0000", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0389000000000002e-05 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379642+0000", 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8748999999999998e-05 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:03.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:03.079 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:03.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:03.081 INFO:teuthology.orchestra.run.smithi165.stdout:2794 write 0x8257a00 thru 0x82620bc (0xa6bd bytes) 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2796 trunc from 0x8262200 to 0x4aab400 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2797 trunc from 0x4aab400 to 0x7389400 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2799 write 0xc80e00 thru 0xc8ee89 (0xe08a bytes) 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2803 punch from 0x56c8600 to 0x56d6400, (0xde00 bytes) 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2804 write 0x63bc600 thru 0x63c6e89 (0xa88a bytes) 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2806 write 0xb90a000 thru 0xb90c516 (0x2517 bytes) 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2807 trunc from 0xb90c600 to 0x8a65400 2024-09-17T21:47:03.082 INFO:teuthology.orchestra.run.smithi165.stdout:2808 write 0x56e4800 thru 0x56f0ffe (0xc7ff bytes) 2024-09-17T21:47:03.082 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:03.077+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:03.083 INFO:teuthology.orchestra.run.smithi165.stdout:2809 write 0x963d200 thru 0x9643538 (0x6339 bytes) 2024-09-17T21:47:03.085 INFO:teuthology.orchestra.run.smithi165.stdout:2810 read 0x6d20000 thru 0x6d2bb73 (0xbb74 bytes) 2024-09-17T21:47:03.085 INFO:teuthology.orchestra.run.smithi165.stdout:2812 write 0x38a5000 thru 0x38a52b3 (0x2b4 bytes) 2024-09-17T21:47:03.088 INFO:teuthology.orchestra.run.smithi165.stdout:2813 punch from 0x3925a00 to 0x392b800, (0x5e00 bytes) 2024-09-17T21:47:03.095 INFO:teuthology.orchestra.run.smithi165.stdout:2815 trunc from 0x9643600 to 0x1dae600 2024-09-17T21:47:03.119 INFO:teuthology.orchestra.run.smithi165.stdout:2816 write 0x17ca00 thru 0x1869b6 (0x9fb7 bytes) 2024-09-17T21:47:03.144 INFO:teuthology.orchestra.run.smithi165.stdout:2819 write 0x5390c00 thru 0x53980f6 (0x74f7 bytes) 2024-09-17T21:47:03.152 INFO:teuthology.orchestra.run.smithi165.stdout:2820 trunc from 0x5398200 to 0x1540400 2024-09-17T21:47:03.171 INFO:teuthology.orchestra.run.smithi165.stdout:2824 punch from 0xf3b400 to 0xf41200, (0x5e00 bytes) 2024-09-17T21:47:03.173 INFO:teuthology.orchestra.run.smithi165.stdout:2826 clone 23 order 20 su 32768 sc 10 2024-09-17T21:47:03.181 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:03.178+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:03.240 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:03.240 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:03.240 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:03.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:03.281 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:03.279+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:03.356 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:03.356 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:03.357 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:03.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:03.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:03.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:03.382 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:03.380+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:03.483 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:03.480+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:03.483 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:03.483 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:03.501 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:03.501 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:03.512 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:03.517 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:03.517 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:03.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:03.535 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:03.535 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:03.552 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:03.552 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:03.569 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:03.570 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:03.622 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:03.619+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:03.643 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:03.643 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:03.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:03.692 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:03.689+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:03.786 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:03.783+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:03.798 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:03.798 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:03.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:47:03.911 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:03.908+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:03.926 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone22 intact 2024-09-17T21:47:03.960 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:03.960 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:03.969 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:04.010 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:04.007+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:04.086 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.086 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.088 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:04.086+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:04.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:04.189 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:04.185+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:04.210 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-17T21:47:04.244 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.244 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:04.296 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:04.294+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:04.374 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.375 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:04.390 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:04.387+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:04.491 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:04.489+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:04.553 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.553 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.564 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:04.605 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:04.603+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:04.693 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:04.691+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:04.718 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.718 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:04.811 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:04.809+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:04.874 INFO:teuthology.orchestra.run.smithi165.stdout:2827 punch from 0xea7c00 to 0xeaa800, (0x2c00 bytes) 2024-09-17T21:47:04.876 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:04.876 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:04.885 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:04.889 INFO:teuthology.orchestra.run.smithi165.stdout:2831 trunc from 0x1540400 to 0x39f3c00 2024-09-17T21:47:04.904 INFO:teuthology.orchestra.run.smithi165.stdout:2834 read 0x556600 thru 0x558074 (0x1a75 bytes) 2024-09-17T21:47:04.905 INFO:teuthology.orchestra.run.smithi165.stdout:2835 read 0x738a00 thru 0x747e6b (0xf46c bytes) 2024-09-17T21:47:04.905 INFO:teuthology.orchestra.run.smithi165.stdout:2836 punch from 0xd22400 to 0xd22e00, (0xa00 bytes) 2024-09-17T21:47:04.906 INFO:teuthology.orchestra.run.smithi165.stdout:2838 punch from 0x26ee600 to 0x26f6a00, (0x8400 bytes) 2024-09-17T21:47:04.924 INFO:teuthology.orchestra.run.smithi165.stdout:2839 write 0xb39e400 thru 0xb3ac7c8 (0xe3c9 bytes) 2024-09-17T21:47:04.926 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:04.924+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:04.942 INFO:teuthology.orchestra.run.smithi165.stdout:2841 write 0xdfcde00 thru 0xdfd930f (0xb510 bytes) 2024-09-17T21:47:04.943 INFO:teuthology.orchestra.run.smithi165.stdout:2842 read 0x1b2ba00 thru 0x1b3b8fd (0xfefe bytes) 2024-09-17T21:47:04.943 INFO:teuthology.orchestra.run.smithi165.stdout:2844 read 0x98ca200 thru 0x98cb08f (0xe90 bytes) 2024-09-17T21:47:04.943 INFO:teuthology.orchestra.run.smithi165.stdout:2845 write 0xa82aa00 thru 0xa83603d (0xb63e bytes) 2024-09-17T21:47:04.943 INFO:teuthology.orchestra.run.smithi165.stdout:2846 write 0xdadc000 thru 0xdae1891 (0x5892 bytes) 2024-09-17T21:47:04.943 INFO:teuthology.orchestra.run.smithi165.stdout:2847 read 0x580f000 thru 0x580f647 (0x648 bytes) 2024-09-17T21:47:04.944 INFO:teuthology.orchestra.run.smithi165.stdout:2848 read 0x954f400 thru 0x95526e3 (0x32e4 bytes) 2024-09-17T21:47:04.944 INFO:teuthology.orchestra.run.smithi165.stdout:2850 write 0x8ff9400 thru 0x900404a (0xac4b bytes) 2024-09-17T21:47:04.944 INFO:teuthology.orchestra.run.smithi165.stdout:2853 punch from 0x1e61a00 to 0x1e6a600, (0x8c00 bytes) 2024-09-17T21:47:04.965 INFO:teuthology.orchestra.run.smithi165.stdout:2854 write 0xe7ac000 thru 0xe7bb434 (0xf435 bytes) 2024-09-17T21:47:04.966 INFO:teuthology.orchestra.run.smithi165.stdout:2855 trunc from 0xe7bb600 to 0xdcbfa00 2024-09-17T21:47:04.986 INFO:teuthology.orchestra.run.smithi165.stdout:2858 read 0x7434600 thru 0x7437cc7 (0x36c8 bytes) 2024-09-17T21:47:04.987 INFO:teuthology.orchestra.run.smithi165.stdout:2859 clone 24 order 23 su 1048576 sc 6 2024-09-17T21:47:04.996 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:04.993+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:05.035 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:05.036 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:05.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:05.097 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:05.094+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:05.150 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:05.151 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:05.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:05.200 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:05.198+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:05.299 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:05.295+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:05.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:05.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:05.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:05.400 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:05.396+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:05.427 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:05.427 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:05.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:05.500 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:05.498+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:05.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:05.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:05.601 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:05.598+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:05.701 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:05.698+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:05.705 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:05.705 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:05.723 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:05.723 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:05.739 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:221 3.1f 3:fc4c7aa4:::rbd_data.1112d1429a58.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 2297856~19456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.457649748000001, 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095307840000000005, 2024-09-17T21:47:05.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 221 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282794+0000", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.282796+0000", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.55e-06 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285729+0000", 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002933569 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285744+0000", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4786e-05 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285749+0000", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2329999999999997e-06 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285798+0000", 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8548999999999999e-05 2024-09-17T21:47:05.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.285925+0000", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012678400000000001 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.286232+0000", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030774099999999999 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.291318+0000", 2024-09-17T21:47:05.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0050855379999999997 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292256+0000", 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00093844800000000004 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292295+0000", 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8590000000000002e-05 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.292325+0000", 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9995999999999999e-05 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4480.0:340 3.1b 3:dfdb8a4f:::rbd_data.1112d1429a58.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13631488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 57.435722943999998, 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063015969999999999, 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4480", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1333364625", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 340 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304721+0000", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.304722+0000", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0999999999999998e-07 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306087+0000", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00136553 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306101+0000", 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3953e-05 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306109+0000", 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3729999999999993e-06 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306174+0000", 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4338000000000006e-05 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306323+0000", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000149598 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.306607+0000", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028390000000000002 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.309257+0000", 2024-09-17T21:47:05.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026498910000000001 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310920+0000", 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0016626480000000001 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.310981+0000", 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1365000000000006e-05 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:08.311022+0000", 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1291000000000002e-05 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1181 3.1 3:8018393d:::rbd_header.11125c054388:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.084334908000002, 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039052739999999998, 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:05.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1181 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656109+0000", 2024-09-17T21:47:05.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656111+0000", 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656122+0000", 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1385e-05 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656142+0000", 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9714000000000001e-05 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656153+0000", 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1287000000000001e-05 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656257+0000", 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010381499999999999 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.656266+0000", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8640000000000002e-06 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658258+0000", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019923480000000001 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658316+0000", 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7639000000000002e-05 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.658532+0000", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216479 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659106+0000", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057333000000000004 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659912+0000", 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00080621499999999999 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.659970+0000", 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8372999999999998e-05 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.660014+0000", 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3775e-05 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:360 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24489984~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.579396054, 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032864750000000001, 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 360 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161048+0000", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161049+0000", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0270000000000001e-06 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161052+0000", 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9739999999999998e-06 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161055+0000", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6079999999999998e-06 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164242+0000", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031871830000000001 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164283+0000", 2024-09-17T21:47:05.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1600000000000002e-05 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164334+0000", 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0881999999999998e-05 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:361 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24420352~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.579386442999997, 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034035810000000001, 2024-09-17T21:47:05.767 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 361 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161057+0000", 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0200000000000001e-07 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161058+0000", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000001e-07 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161062+0000", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.128e-06 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161064+0000", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5119999999999998e-06 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164358+0000", 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032940819999999998 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164402+0000", 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3948e-05 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164461+0000", 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8797e-05 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:362 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24469504~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.579375612, 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035274899999999999, 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161068+0000", 2024-09-17T21:47:05.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161072+0000", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5990000000000002e-06 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161076+0000", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9389999999999999e-06 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161079+0000", 2024-09-17T21:47:05.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7300000000000001e-06 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164490+0000", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034114420000000002 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164541+0000", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1180999999999997e-05 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164596+0000", 2024-09-17T21:47:05.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4388999999999997e-05 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:363 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24424448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.579362021000001, 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0036505470000000001, 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 363 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:05.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161082+0000", 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161084+0000", 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5239999999999999e-06 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161087+0000", 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161089+0000", 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164621+0000", 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0035319700000000002 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164670+0000", 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8837e-05 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164732+0000", 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2353000000000004e-05 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:364 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24473600~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.579351670999998, 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0037536079999999999, 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 364 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161092+0000", 2024-09-17T21:47:05.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161093+0000", 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2099999999999995e-07 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161096+0000", 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0819999999999999e-06 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.161099+0000", 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6879999999999999e-06 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164749+0000", 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003649663 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164791+0000", 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2345000000000002e-05 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.164846+0000", 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4684e-05 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:463 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24768512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.569912598999998, 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032941569999999998, 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 463 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170531+0000", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170532+0000", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0600000000000001e-07 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170533+0000", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.051e-06 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170536+0000", 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170538+0000", 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4339999999999999e-06 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173720+0000", 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031821430000000001 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173769+0000", 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789000000000003e-05 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173825+0000", 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6450999999999999e-05 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:464 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24645632~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.569773390000002, 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032981960000000002, 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 464 2024-09-17T21:47:05.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170670+0000", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170671+0000", 2024-09-17T21:47:05.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170673+0000", 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4269999999999998e-06 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170679+0000", 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-06 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170682+0000", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1650000000000002e-06 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173859+0000", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0031773560000000001 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173905+0000", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5655000000000002e-05 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173969+0000", 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3631000000000001e-05 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:465 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24743936~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.569756400000003, 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034171140000000002, 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 465 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170687+0000", 2024-09-17T21:47:05.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170688+0000", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1199999999999999e-07 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170689+0000", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.212e-06 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170692+0000", 2024-09-17T21:47:05.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1769999999999998e-06 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170703+0000", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1078e-05 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.173999+0000", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032957170000000001 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174044+0000", 2024-09-17T21:47:05.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5577000000000001e-05 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174105+0000", 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0140999999999999e-05 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:466 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24694784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.569735401999999, 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003526937, 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 466 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170708+0000", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170709+0000", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-07 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170710+0000", 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.469e-06 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170714+0000", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.568e-06 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.170717+0000", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8899999999999999e-06 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174125+0000", 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034079969999999998 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174179+0000", 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4503000000000003e-05 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.174235+0000", 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6233999999999997e-05 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:514 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24834048~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.564876839, 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033585059999999998, 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 514 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:05.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175567+0000", 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-07 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175568+0000", 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.102e-06 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175572+0000", 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6229999999999999e-06 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175574+0000", 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.52e-06 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178824+0000", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032500010000000002 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178870+0000", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5537999999999997e-05 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178925+0000", 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5501999999999998e-05 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:515 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:7 [read 24883200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.564866506999998, 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034717699999999999, 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 515 2024-09-17T21:47:05.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175577+0000", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175578+0000", 2024-09-17T21:47:05.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175579+0000", 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175582+0000", 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-06 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.175585+0000", 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5579999999999999e-06 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178947+0000", 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033625299999999999 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.178990+0000", 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2651999999999998e-05 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:12.179049+0000", 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9197999999999998e-05 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.057622610999999, 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:05.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:05.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:05.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.025347635999999, 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:05.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:05.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.025108867999997, 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:05.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:05.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.880399253, 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:05.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:05.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:05.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5145.0:362 3.6 3:64a48ca3:::rbd_data.1112b4c08b07.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 7016448~57344,write 7602176~36864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.3673371000000003, 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065350929999999996, 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5145", 2024-09-17T21:47:05.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/187823783", 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 362 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373107+0000", 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.373108+0000", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2669999999999999e-06 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374613+0000", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0015053499999999999 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374626+0000", 2024-09-17T21:47:05.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2616000000000001e-05 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374634+0000", 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1829999999999994e-06 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374681+0000", 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6828e-05 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.374778+0000", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7036999999999999e-05 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.375069+0000", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029081000000000002 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.378468+0000", 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0033987090000000002 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379563+0000", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001095155 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379603+0000", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0389000000000002e-05 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:58.379642+0000", 2024-09-17T21:47:05.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8748999999999998e-05 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5232.0:779 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,truncate 5957120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e74)", 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:03.107663+0000", 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 2.632780941, 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0032985200000000001, 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5232", 2024-09-17T21:47:05.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/223075346", 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 779 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.107663+0000", 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.107663+0000", 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.107664+0000", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.06e-07 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108326+0000", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00066245900000000001 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108339+0000", 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2805e-05 2024-09-17T21:47:05.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108347+0000", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4950000000000008e-06 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108416+0000", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8264999999999998e-05 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108527+0000", 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000111319 2024-09-17T21:47:05.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.108824+0000", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029682999999999999 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.110290+0000", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001466261 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.110863+0000", 2024-09-17T21:47:05.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00057277700000000001 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.110908+0000", 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.562e-05 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:03.110961+0000", 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2982999999999997e-05 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:05.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:05.844 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:05.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:05.846 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:05.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:05.846 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:05.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:05.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:05.847 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:05.847 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:05.841+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:05.864 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0] out_osds: [4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:47:05.864 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:05.864 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-17T21:47:05.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-17T21:47:05.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:47:05.944 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone23 from 0xdcbfa00 (overlap 0x1540400) to 0x660000 2024-09-17T21:47:05.946 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:05.943+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:06.038 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:06.038 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.047 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:06.044+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:06.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:06.148 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:06.145+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:06.162 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.162 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:06.163 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:06.163 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:06.248 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:06.245+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:06.273 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:47:06.270+0000 7f0c28614640 -1 mon.a@0(leader).osd e76 definitely_dead 0 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:06.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381268458999998, 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381258158000001, 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:06.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381242084, 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:06.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:06.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381230871, 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:06.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:06.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381213432999999, 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:06.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:06.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381202563000002, 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.381007685999997, 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:06.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.379303016000001, 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:06.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:06.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.379292378000002, 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:06.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:06.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.379282285000002, 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:06.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.379272166, 2024-09-17T21:47:06.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:06.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:06.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.377141936000001, 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:06.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.377132205000002, 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:06.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:06.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.376348845999999, 2024-09-17T21:47:06.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:06.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.376333275999997, 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.376240373999998, 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.375745754999997, 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.375601709999998, 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.374916272999997, 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:06.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.374553882999997, 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:06.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:06.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.378 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-17T21:47:06.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:06.379 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:06.375+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:06.471 INFO:teuthology.orchestra.run.smithi165.stdout:2862 read 0x29c800 thru 0x29f660 (0x2e61 bytes) 2024-09-17T21:47:06.472 INFO:teuthology.orchestra.run.smithi165.stdout:2865 punch from 0x633f600 to 0x634c600, (0xd000 bytes) 2024-09-17T21:47:06.478 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:06.476+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:06.480 INFO:teuthology.orchestra.run.smithi165.stdout:2868 write 0xd492800 thru 0xd4a10af (0xe8b0 bytes) 2024-09-17T21:47:06.493 INFO:teuthology.orchestra.run.smithi165.stdout:2872 trunc from 0xdcbfa00 to 0x2e9b400 2024-09-17T21:47:06.535 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:06.535 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.537 INFO:teuthology.orchestra.run.smithi165.stdout:2875 trunc from 0x2e9b400 to 0xc414000 2024-09-17T21:47:06.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:06.546 INFO:teuthology.orchestra.run.smithi165.stdout:2876 punch from 0x137a800 to 0x1389e00, (0xf600 bytes) 2024-09-17T21:47:06.549 INFO:teuthology.orchestra.run.smithi165.stdout:2877 write 0x2309000 thru 0x2310efc (0x7efd bytes) 2024-09-17T21:47:06.550 INFO:teuthology.orchestra.run.smithi165.stdout:2878 punch from 0x860f400 to 0x861bc00, (0xc800 bytes) 2024-09-17T21:47:06.551 INFO:teuthology.orchestra.run.smithi165.stdout:2880 write 0x669c400 thru 0x66a6b2b (0xa72c bytes) 2024-09-17T21:47:06.551 INFO:teuthology.orchestra.run.smithi165.stdout:2881 read 0x5630200 thru 0x5635e0c (0x5c0d bytes) 2024-09-17T21:47:06.551 INFO:teuthology.orchestra.run.smithi165.stdout:2883 write 0x55d4c00 thru 0x55dd43a (0x883b bytes) 2024-09-17T21:47:06.551 INFO:teuthology.orchestra.run.smithi165.stdout:2884 write 0x6951a00 thru 0x695c754 (0xad55 bytes) 2024-09-17T21:47:06.584 INFO:teuthology.orchestra.run.smithi165.stdout:2885 write 0xde34400 thru 0xde3679d (0x239e bytes) 2024-09-17T21:47:06.589 INFO:teuthology.orchestra.run.smithi165.stdout:2886 punch from 0x3250400 to 0x325be00, (0xba00 bytes) 2024-09-17T21:47:06.590 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:06.587+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:06.591 INFO:teuthology.orchestra.run.smithi165.stdout:2888 write 0x5193400 thru 0x519ab32 (0x7733 bytes) 2024-09-17T21:47:06.591 INFO:teuthology.orchestra.run.smithi165.stdout:2892 read 0x72d5000 thru 0x72d9afe (0x4aff bytes) 2024-09-17T21:47:06.592 INFO:teuthology.orchestra.run.smithi165.stdout:2895 punch from 0xc241800 to 0xc24f400, (0xdc00 bytes) 2024-09-17T21:47:06.592 INFO:teuthology.orchestra.run.smithi165.stdout:2896 read 0xdd62000 thru 0xdd6f419 (0xd41a bytes) 2024-09-17T21:47:06.592 INFO:teuthology.orchestra.run.smithi165.stdout:2898 read 0xd632000 thru 0xd63d7ef (0xb7f0 bytes) 2024-09-17T21:47:06.593 INFO:teuthology.orchestra.run.smithi165.stdout:2900 read 0x576a800 thru 0x576c52a (0x1d2b bytes) 2024-09-17T21:47:06.593 INFO:teuthology.orchestra.run.smithi165.stdout:2903 write 0x2ae5a00 thru 0x2ae79ec (0x1fed bytes) 2024-09-17T21:47:06.593 INFO:teuthology.orchestra.run.smithi165.stdout:2904 write 0xd685a00 thru 0xd68a302 (0x4903 bytes) 2024-09-17T21:47:06.593 INFO:teuthology.orchestra.run.smithi165.stdout:2906 write 0x2737200 thru 0x273f482 (0x8283 bytes) 2024-09-17T21:47:06.593 INFO:teuthology.orchestra.run.smithi165.stdout:2908 punch from 0x8889800 to 0x8889c00, (0x400 bytes) 2024-09-17T21:47:06.594 INFO:teuthology.orchestra.run.smithi165.stdout:2909 read 0x60ae600 thru 0x60be0d7 (0xfad8 bytes) 2024-09-17T21:47:06.600 INFO:teuthology.orchestra.run.smithi165.stdout:2911 read 0x8cc2800 thru 0x8cc3564 (0xd65 bytes) 2024-09-17T21:47:06.600 INFO:teuthology.orchestra.run.smithi165.stdout:2912 read 0x7043000 thru 0x704507f (0x2080 bytes) 2024-09-17T21:47:06.609 INFO:teuthology.orchestra.run.smithi165.stdout:2913 trunc from 0xde36800 to 0x6999000 2024-09-17T21:47:06.638 INFO:teuthology.orchestra.run.smithi165.stdout:2916 read 0x438ce00 thru 0x4390da8 (0x3fa9 bytes) 2024-09-17T21:47:06.639 INFO:teuthology.orchestra.run.smithi165.stdout:2918 read 0x3427e00 thru 0x3435c76 (0xde77 bytes) 2024-09-17T21:47:06.648 INFO:teuthology.orchestra.run.smithi165.stdout:2920 trunc from 0x6999000 to 0xe778600 2024-09-17T21:47:06.659 INFO:teuthology.orchestra.run.smithi165.stdout:2921 punch from 0xb90600 to 0xb9a200, (0x9c00 bytes) 2024-09-17T21:47:06.664 INFO:teuthology.orchestra.run.smithi165.stdout:2922 trunc from 0xe778600 to 0xbbeb800 2024-09-17T21:47:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.676 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:06.676 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:06.676 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:06.676 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.679 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:06.677+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:06.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:06.685 INFO:teuthology.orchestra.run.smithi165.stdout:2923 trunc from 0xbbeb800 to 0x2b2be00 2024-09-17T21:47:06.738 INFO:teuthology.orchestra.run.smithi165.stdout:2927 write 0x71fc600 thru 0x7203cd4 (0x76d5 bytes) 2024-09-17T21:47:06.746 INFO:teuthology.orchestra.run.smithi165.stdout:2929 trunc from 0x7203e00 to 0xd81dc00 2024-09-17T21:47:06.767 INFO:teuthology.orchestra.run.smithi165.stdout:2930 write 0x862d000 thru 0x863666c (0x966d bytes) 2024-09-17T21:47:06.767 INFO:teuthology.orchestra.run.smithi165.stdout:2931 write 0xa9ef400 thru 0xa9f6962 (0x7563 bytes) 2024-09-17T21:47:06.767 INFO:teuthology.orchestra.run.smithi165.stdout:2932 write 0x57a1e00 thru 0x57a1fe0 (0x1e1 bytes) 2024-09-17T21:47:06.767 INFO:teuthology.orchestra.run.smithi165.stdout:2933 write 0xa5cba00 thru 0xa5cbbd7 (0x1d8 bytes) 2024-09-17T21:47:06.768 INFO:teuthology.orchestra.run.smithi165.stdout:2935 punch from 0xc762600 to 0xc768a00, (0x6400 bytes) 2024-09-17T21:47:06.768 INFO:teuthology.orchestra.run.smithi165.stdout:2936 write 0x37c4000 thru 0x37d1a08 (0xda09 bytes) 2024-09-17T21:47:06.768 INFO:teuthology.orchestra.run.smithi165.stdout:2937 read 0xbf46400 thru 0xbf563cb (0xffcc bytes) 2024-09-17T21:47:06.769 INFO:teuthology.orchestra.run.smithi165.stdout:2940 read 0x1c28e00 thru 0x1c2e766 (0x5967 bytes) 2024-09-17T21:47:06.769 INFO:teuthology.orchestra.run.smithi165.stdout:2943 read 0x3e43e00 thru 0x3e445d7 (0x7d8 bytes) 2024-09-17T21:47:06.769 INFO:teuthology.orchestra.run.smithi165.stdout:2944 read 0xcfe1600 thru 0xcfe8f05 (0x7906 bytes) 2024-09-17T21:47:06.769 INFO:teuthology.orchestra.run.smithi165.stdout:2945 punch from 0x13fe800 to 0x1404200, (0x5a00 bytes) 2024-09-17T21:47:06.769 INFO:teuthology.orchestra.run.smithi165.stdout:2948 punch from 0xce0f400 to 0xce17600, (0x8200 bytes) 2024-09-17T21:47:06.773 INFO:teuthology.orchestra.run.smithi165.stdout:2949 trunc from 0xd81dc00 to 0x9ab7600 2024-09-17T21:47:06.780 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:06.777+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:06.800 INFO:teuthology.orchestra.run.smithi165.stdout:2950 read 0x374fc00 thru 0x3750002 (0x403 bytes) 2024-09-17T21:47:06.801 INFO:teuthology.orchestra.run.smithi165.stdout:2951 read 0x6502c00 thru 0x650ce33 (0xa234 bytes) 2024-09-17T21:47:06.801 INFO:teuthology.orchestra.run.smithi165.stdout:2953 read 0x6f0a600 thru 0x6f172be (0xccbf bytes) 2024-09-17T21:47:06.801 INFO:teuthology.orchestra.run.smithi165.stdout:2956 read 0x7b6cc00 thru 0x7b6d1d5 (0x5d6 bytes) 2024-09-17T21:47:06.802 INFO:teuthology.orchestra.run.smithi165.stdout:2961 trunc from 0x9ab7600 to 0xa056400 2024-09-17T21:47:06.812 INFO:teuthology.orchestra.run.smithi165.stdout:2964 punch from 0x5e7000 to 0x5eec00, (0x7c00 bytes) 2024-09-17T21:47:06.827 INFO:teuthology.orchestra.run.smithi165.stdout:2965 write 0xb8e7200 thru 0xb8f2a56 (0xb857 bytes) 2024-09-17T21:47:06.829 INFO:teuthology.orchestra.run.smithi165.stdout:2967 punch from 0x67b6000 to 0x67bdc00, (0x7c00 bytes) 2024-09-17T21:47:06.829 INFO:teuthology.orchestra.run.smithi165.stdout:2968 write 0x1f5bc00 thru 0x1f62c01 (0x7002 bytes) 2024-09-17T21:47:06.831 INFO:teuthology.orchestra.run.smithi165.stdout:2970 write 0x5cb800 thru 0x5da2f5 (0xeaf6 bytes) 2024-09-17T21:47:06.831 INFO:teuthology.orchestra.run.smithi165.stdout:2973 punch from 0xa559600 to 0xa560800, (0x7200 bytes) 2024-09-17T21:47:06.831 INFO:teuthology.orchestra.run.smithi165.stdout:2974 write 0xad53800 thru 0xad56f5f (0x3760 bytes) 2024-09-17T21:47:06.831 INFO:teuthology.orchestra.run.smithi165.stdout:2975 punch from 0x9d40000 to 0x9d49c00, (0x9c00 bytes) 2024-09-17T21:47:06.837 INFO:teuthology.orchestra.run.smithi165.stdout:2977 trunc from 0xb8f2c00 to 0x641ce00 2024-09-17T21:47:06.856 INFO:teuthology.orchestra.run.smithi165.stdout:2978 read 0x5991600 thru 0x599b5a3 (0x9fa4 bytes) 2024-09-17T21:47:06.857 INFO:teuthology.orchestra.run.smithi165.stdout:2981 read 0x5260a00 thru 0x526a1f5 (0x97f6 bytes) 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.045327499000003, 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:47:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:47:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:47:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.035676535, 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:47:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.951730666000003, 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:47:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:47:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.950268530999999, 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.947657413000002, 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:47:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:47:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.883542955999999, 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.882814734, 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:47:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.549237597999998, 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:47:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.982479095999999, 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.974956954, 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1128 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.864667882, 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099832400000000009, 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1128 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990586+0000", 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.125e-06 2024-09-17T21:47:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990593+0000", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6919999999999996e-06 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990598+0000", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9830000000000004e-06 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000414+0000", 2024-09-17T21:47:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098157309999999994 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000481+0000", 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7303000000000003e-05 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000567+0000", 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5923999999999999e-05 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1129 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.86464733100000002, 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010143592999999999, 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1129 2024-09-17T21:47:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990605+0000", 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990607+0000", 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990613+0000", 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990618+0000", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000596+0000", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099777809999999998 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000662+0000", 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5891999999999999e-05 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000748+0000", 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6372999999999994e-05 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1224 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.85779789799999995, 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010061038, 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1224 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997457+0000", 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3499999999999999e-06 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998930+0000", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014735079999999999 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998947+0000", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6977999999999999e-05 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007338+0000", 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083907679999999998 2024-09-17T21:47:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007417+0000", 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9008e-05 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007515+0000", 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7958e-05 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1255 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.85582198600000003, 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010112939, 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1255 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:06.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999432+0000", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249999999999999e-06 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999439+0000", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999445+0000", 2024-09-17T21:47:06.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2730000000000002e-06 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009368+0000", 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099234240000000001 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009446+0000", 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8209999999999998e-05 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009543+0000", 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6353000000000004e-05 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1256 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.85580052100000004, 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010198560000000001, 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1256 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999452+0000", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999453+0000", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.945e-06 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999460+0000", 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7789999999999998e-06 2024-09-17T21:47:06.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999469+0000", 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6309999999999992e-06 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009556+0000", 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010086927000000001 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009597+0000", 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1038999999999998e-05 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009650+0000", 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2831999999999998e-05 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1288 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:06.935 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.85278420200000005, 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010050948000000001, 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1288 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002468+0000", 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002470+0000", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.384e-06 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002477+0000", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002482+0000", 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1089999999999997e-06 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012369+0000", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098866030000000008 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012434+0000", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5512999999999994e-05 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012518+0000", 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4089000000000001e-05 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1307 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84476695000000002, 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099939750000000004, 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1307 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010488+0000", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0240000000000002e-06 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010495+0000", 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010508+0000", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3410999999999999e-05 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017483+0000", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069745320000000003 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017570+0000", 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7057000000000004e-05 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017970+0000", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040009600000000001 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020303+0000", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023332729999999999 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020439+0000", 2024-09-17T21:47:06.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135783 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020462+0000", 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2469000000000001e-05 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020479+0000", 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7144999999999999e-05 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1309 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.844734553, 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010826840000000001, 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1309 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010518+0000", 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:47:06.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010521+0000", 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.207e-06 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010528+0000", 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010533+0000", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1930000000000001e-06 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018747+0000", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082139880000000002 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018835+0000", 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7560999999999997e-05 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.019193+0000", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035790399999999998 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020506+0000", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013139099999999999 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021255+0000", 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074901999999999998 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021305+0000", 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230999999999997e-05 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021344+0000", 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9252000000000003e-05 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1311 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84471105899999999, 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012406011, 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1311 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010544+0000", 2024-09-17T21:47:06.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.977e-06 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010551+0000", 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7240000000000003e-06 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010556+0000", 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0989999999999998e-06 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020524+0000", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.009968457 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020568+0000", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4132999999999999e-05 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020784+0000", 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021525299999999999 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022838+0000", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020542030000000001 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022848+0000", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9879999999999999e-06 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022894+0000", 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5803000000000002e-05 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022947+0000", 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2896999999999999e-05 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1312 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.84232850599999998, 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011273178999999999, 2024-09-17T21:47:06.957 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1312 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012924+0000", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012937+0000", 2024-09-17T21:47:06.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2662e-05 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012950+0000", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012958+0000", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9140000000000008e-06 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021388+0000", 2024-09-17T21:47:06.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084298759999999993 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021609+0000", 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221256 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021953+0000", 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034352200000000002 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:06.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022955+0000", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010025909999999999 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024087+0000", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011314109999999999 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024148+0000", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1001000000000001e-05 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:06.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024196+0000", 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8474999999999999e-05 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:06.963 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.4. 2024-09-17T21:47:06.964 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:47:06.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:06.966 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-17T21:47:06.967 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:06.962+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:07.065 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:07.062+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:07.074 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:07.074 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:07.166 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:07.164+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:07.189 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.189 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:07.189 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:07.189 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:07.189 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:07.268 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:07.265+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.185432493999997, 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:47:07.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:47:07.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.524447205000001, 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.506875813999997, 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.360 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.506842120999998, 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:47:07.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.504000818000002, 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:47:07.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:47:07.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503923507000003, 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503901114000001, 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:07.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:47:07.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:47:07.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503879992999998, 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503859249000001, 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:47:07.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503444604999999, 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:07.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:47:07.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.503409414000004, 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.396 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:47:07.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:47:07.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.400 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.502517951000002, 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:47:07.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:07.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.502151882, 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:47:07.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.448844663999999, 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:47:07.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:47:07.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.448808892000002, 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.444670520000003, 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:47:07.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:47:07.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.439476798999998, 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:47:07.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:47:07.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.046970205999997, 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:07.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:47:07.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.475670473000001, 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:07.438 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:47:07.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:47:07.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.470123109999999, 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:47:07.444 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:47:07.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:47:07.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.450 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.452 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:07.452 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:07.449+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:07.552 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:07.550+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:07.598 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:07.599 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:07.654 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:07.651+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:07.715 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:07.715 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:07.715 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.723 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:07.767 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:07.765+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.878384521000001, 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.876359469000001, 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:07.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:07.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.862373413, 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:07.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:07.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.860412205999999, 2024-09-17T21:47:07.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:07.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:07.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:07.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.858378848999998, 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:07.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:07.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.856400931000003, 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:07.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:07.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.856378137, 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:07.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:07.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.850469074000003, 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:07.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:07.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:07.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.848438014999999, 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:07.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.848411441000003, 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:07.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:07.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.847801648000001, 2024-09-17T21:47:07.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:07.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:07.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.847784062000002, 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:07.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:07.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.892 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.847769104999998, 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:07.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.132087617000003, 2024-09-17T21:47:07.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:07.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.132056996000003, 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:07.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.132032377999998, 2024-09-17T21:47:07.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:07.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.132010430999998, 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:07.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:07.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.131988960999998, 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:07.922 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:07.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.090572158000001, 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:07.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:07.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.089527697999998, 2024-09-17T21:47:07.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:07.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:07.938 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:07.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:07.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:07.941 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:07.937+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:08.040 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:08.038+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:08.073 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:08.089 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:08.089 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:08.090 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:08.090 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:08.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:08.106 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:08.107 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:08.124 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:08.124 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:08.141 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:08.137+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:08.141 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:08.141 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:08.158 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:08.159 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:08.241 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:08.239+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:08.245 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.245 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.254 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:08.342 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:08.340+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:08.361 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.361 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:08.443 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:08.440+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.478 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:08.544 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:08.542+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:08.634 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.635 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:08.644 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:08.642+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:08.745 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:08.742+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:08.750 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.750 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:08.846 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:08.842+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:08.867 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:08.867 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:08.882 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:08.947 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:08.944+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:08.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:47:09.031 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.031 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:09.048 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:09.045+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:09.149 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:09.146+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:09.173 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.174 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:09.249 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:09.247+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:09.290 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.290 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:09.350 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:09.347+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:09.406 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.406 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.415 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:09.456 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:09.453+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:09.542 INFO:teuthology.orchestra.run.smithi165.stdout:2982 write 0xb34f200 thru 0xb357b21 (0x8922 bytes) 2024-09-17T21:47:09.580 INFO:teuthology.orchestra.run.smithi165.stdout:2983 write 0x3655c00 thru 0x3661fa0 (0xc3a1 bytes) 2024-09-17T21:47:09.580 INFO:teuthology.orchestra.run.smithi165.stdout:2985 read 0xa97e800 thru 0xa97fe6f (0x1670 bytes) 2024-09-17T21:47:09.580 INFO:teuthology.orchestra.run.smithi165.stdout:2986 write 0xe709600 thru 0xe713fc0 (0xa9c1 bytes) 2024-09-17T21:47:09.581 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.581 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:09.582 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:09.576+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:09.587 INFO:teuthology.orchestra.run.smithi165.stdout:2987 trunc from 0xe714000 to 0x4585000 2024-09-17T21:47:09.610 INFO:teuthology.orchestra.run.smithi165.stdout:2988 punch from 0x1547e00 to 0x1549200, (0x1400 bytes) 2024-09-17T21:47:09.619 INFO:teuthology.orchestra.run.smithi165.stdout:2990 punch from 0x4044600 to 0x4053c00, (0xf600 bytes) 2024-09-17T21:47:09.619 INFO:teuthology.orchestra.run.smithi165.stdout:2993 write 0x444ea00 thru 0x44598c8 (0xaec9 bytes) 2024-09-17T21:47:09.650 INFO:teuthology.orchestra.run.smithi165.stdout:2996 write 0xde61800 thru 0xde67e6b (0x666c bytes) 2024-09-17T21:47:09.657 INFO:teuthology.orchestra.run.smithi165.stdout:2998 punch from 0x9a44400 to 0x9a4b000, (0x6c00 bytes) 2024-09-17T21:47:09.659 INFO:teuthology.orchestra.run.smithi165.stdout:2999 write 0xddee000 thru 0xddf1523 (0x3524 bytes) 2024-09-17T21:47:09.660 INFO:teuthology.orchestra.run.smithi165.stdout:3002 read 0x80b000 thru 0x813fe9 (0x8fea bytes) 2024-09-17T21:47:09.660 INFO:teuthology.orchestra.run.smithi165.stdout:3003 write 0x1b5fa00 thru 0x1b6530d (0x590e bytes) 2024-09-17T21:47:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:3004 read 0x78bb200 thru 0x78bf128 (0x3f29 bytes) 2024-09-17T21:47:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:3006 write 0x8015c00 thru 0x8025a00 (0xfe01 bytes) 2024-09-17T21:47:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:3008 read 0xfbe400 thru 0xfc0ab3 (0x26b4 bytes) 2024-09-17T21:47:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:3009 read 0x982a00 thru 0x987222 (0x4823 bytes) 2024-09-17T21:47:09.669 INFO:teuthology.orchestra.run.smithi165.stdout:3011 trunc from 0xde68000 to 0x8752a00 2024-09-17T21:47:09.681 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:09.678+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:09.702 INFO:teuthology.orchestra.run.smithi165.stdout:3019 trunc from 0x8752a00 to 0x421e00 2024-09-17T21:47:09.718 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.718 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:09.755 INFO:teuthology.orchestra.run.smithi165.stdout:3020 write 0xd866200 thru 0xd8682b5 (0x20b6 bytes) 2024-09-17T21:47:09.758 INFO:teuthology.orchestra.run.smithi165.stdout:3023 read 0x2d7e400 thru 0x2d8d4c3 (0xf0c4 bytes) 2024-09-17T21:47:09.758 INFO:teuthology.orchestra.run.smithi165.stdout:3024 write 0x6e51400 thru 0x6e5c9b8 (0xb5b9 bytes) 2024-09-17T21:47:09.760 INFO:teuthology.orchestra.run.smithi165.stdout:3028 punch from 0x13ac000 to 0x13b0c00, (0x4c00 bytes) 2024-09-17T21:47:09.760 INFO:teuthology.orchestra.run.smithi165.stdout:3029 punch from 0x137b000 to 0x137c000, (0x1000 bytes) 2024-09-17T21:47:09.761 INFO:teuthology.orchestra.run.smithi165.stdout:3034 write 0xb302c00 thru 0xb30f2e3 (0xc6e4 bytes) 2024-09-17T21:47:09.761 INFO:teuthology.orchestra.run.smithi165.stdout:3035 write 0x7035400 thru 0x7036d9b (0x199c bytes) 2024-09-17T21:47:09.761 INFO:teuthology.orchestra.run.smithi165.stdout:3037 write 0x1ba400 thru 0x1bc44b (0x204c bytes) 2024-09-17T21:47:09.765 INFO:teuthology.orchestra.run.smithi165.stdout:3038 trunc from 0xd868400 to 0xb100400 2024-09-17T21:47:09.782 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:09.780+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:09.805 INFO:teuthology.orchestra.run.smithi165.stdout:3039 write 0xbfd9600 thru 0xbfe30b4 (0x9ab5 bytes) 2024-09-17T21:47:09.809 INFO:teuthology.orchestra.run.smithi165.stdout:3040 read 0x1e38a00 thru 0x1e3df36 (0x5537 bytes) 2024-09-17T21:47:09.810 INFO:teuthology.orchestra.run.smithi165.stdout:3041 trunc from 0xbfe3200 to 0xb761200 2024-09-17T21:47:09.838 INFO:teuthology.orchestra.run.smithi165.stdout:3042 trunc from 0xb761200 to 0x29d3a00 2024-09-17T21:47:09.854 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:09.854 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:09.857 INFO:teuthology.orchestra.run.smithi165.stdout:3044 read 0x13f3200 thru 0x13f3cc2 (0xac3 bytes) 2024-09-17T21:47:09.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:09.881 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:47:09.881 INFO:teuthology.orchestra.run.smithi165.stdout:3045 write 0x8519200 thru 0x851a515 (0x1316 bytes) 2024-09-17T21:47:09.905 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:09.903+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:09.984 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:09.982+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:10.025 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:10.026 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:10.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:10.120 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:10.118+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:10.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:10.198 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:10.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:10.214 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:10.211+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:10.292 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:10.290+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:10.295 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:10.312 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:10.328 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:10.329 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:10.345 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:10.346 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:10.362 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:10.363 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:10.371 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:10.371 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:10.371 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:10.380 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:10.380 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:10.383 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:10.423 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:10.421+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:10.488 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:10.485+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.580308488, 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.578283436, 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.564297379999999, 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.562336172999998, 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.560302815999997, 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.558324898000002, 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.558302103999999, 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.552393041000002, 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.550361981999998, 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.550335408000002, 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.549725615, 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.549708029000001, 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 53.549693071999997, 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.834011584000002, 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:10.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:10.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:10.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.833980963000002, 2024-09-17T21:47:10.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:10.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:10.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.833956344999997, 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:10.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:10.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:10.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.833934397999997, 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:10.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:10.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.833912927999997, 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:10.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:10.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:10.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:10.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.792496125, 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:10.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:10.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:10.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.791451664999997, 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:10.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:10.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:10.642 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:10.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:10.644 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:10.641+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:10.745 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:10.742+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:10.793 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:10.793 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:10.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:10.793 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:10.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:10.845 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:10.843+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:10.912 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:10.912 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:10.912 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:10.912 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:10.912 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:10.921 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:10.963 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:10.960+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:11.047 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:11.045+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.097078744999997, 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:11.071 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:11.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.097056252000002, 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.097031991999998, 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.083 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.097006108999999, 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.096981108000001, 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.092 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.096941309000002, 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.096925564000003, 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.097 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.096914028999997, 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:11.101 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:11.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.096903728999997, 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:11.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:11.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:11.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094932485999998, 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:11.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094919337999997, 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:11.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:11.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094908732, 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:11.119 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:11.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094898393999998, 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:11.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094888386999997, 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.128 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:11.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094878436000002, 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:11.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:11.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:11.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.094863128, 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:11.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.092887241, 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:11.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:11.145 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.092876285000003, 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:11.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:11.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.092863796000003, 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.092853333000001, 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:11.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:11.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:11.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:11.158 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:11.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:11.160 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:11.156+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:11.260 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:11.257+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3.5dd07c41 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.38999087700000001, 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.39001119000000001, 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:11.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:11.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:11.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3.5dd07c41 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.38308910600000001, 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.383156946, 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3.5dd07c41 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.38297301500000003, 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.38307559600000002, 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2165 3.1 3.5dd07c41 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.076737936000000007, 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.076873844999999996, 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2165 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196096+0000", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196108+0000", 2024-09-17T21:47:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196130+0000", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1695000000000002e-05 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196140+0000", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196216+0000", 2024-09-17T21:47:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899e-05 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196227+0000", 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1008e-05 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 4 2024-09-17T21:47:11.291 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:11.292 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:11.376 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:11.373+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:11.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:11.489 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:11.487+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:11.563 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:11.560+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:11.597 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.910883165000001, 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.878608190000001, 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.878369421999999, 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.733659806999999, 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:627 3.6 3:6244dd87:::rbd_data.1112bc1b2568.0000000000000019:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6431400739999997, 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030691610000000001, 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 627 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950565+0000", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950567+0000", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950574+0000", 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:47:11.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950580+0000", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950631+0000", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1863999999999997e-05 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950752+0000", 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120596 2024-09-17T21:47:11.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981256+0000", 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030503834000000001 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:745 3.6 3:60bbb74c:::rbd_data.1112bc1b2568.00000000000000de:head [delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6352310479999996, 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023010439000000001, 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 745 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958474+0000", 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958476+0000", 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2299999999999998e-06 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958482+0000", 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958488+0000", 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961234+0000", 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027465929999999999 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961306+0000", 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1521999999999997e-05 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961496+0000", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190053 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.962386+0000", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089013799999999995 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981403+0000", 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019016943000000001 2024-09-17T21:47:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981448+0000", 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5212e-05 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981484+0000", 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5608999999999997e-05 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1205 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.986936+0000", 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6067680549999999, 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012411203000000001, 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.632 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1205 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986936+0000", 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986936+0000", 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986937+0000", 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986939+0000", 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2400000000000002e-06 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986946+0000", 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6120000000000004e-06 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986950+0000", 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8810000000000004e-06 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999194+0000", 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012243864 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999265+0000", 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0740000000000004e-05 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999348+0000", 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2427000000000002e-05 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.636 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1232 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 262144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.986964+0000", 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6067400549999995, 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012568852, 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1232 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986964+0000", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986964+0000", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986965+0000", 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986967+0000", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5100000000000001e-06 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986975+0000", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986980+0000", 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4129999999999997e-06 2024-09-17T21:47:11.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999377+0000", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012396473 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999444+0000", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7069000000000004e-05 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999533+0000", 2024-09-17T21:47:11.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9548000000000004e-05 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1233 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 589824~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.986988+0000", 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6067162399999999, 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012733405999999999, 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1233 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986988+0000", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986988+0000", 2024-09-17T21:47:11.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986989+0000", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.63e-07 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986991+0000", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0930000000000001e-06 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.986998+0000", 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9999999999999999e-06 2024-09-17T21:47:11.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987003+0000", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9989999999999999e-06 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999561+0000", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012558283999999999 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999632+0000", 2024-09-17T21:47:11.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1145999999999999e-05 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999722+0000", 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9420999999999996e-05 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1234 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 917504~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987009+0000", 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6066951539999996, 2024-09-17T21:47:11.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013290959, 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1234 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987009+0000", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987009+0000", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987010+0000", 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:47:11.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987012+0000", 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9989999999999998e-06 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987018+0000", 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.618e-06 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987023+0000", 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0690000000000001e-06 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999751+0000", 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012728092 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999826+0000", 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4357999999999995e-05 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000300+0000", 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000474394 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1235 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6066745239999998, 2024-09-17T21:47:11.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013460730000000001, 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1235 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987032+0000", 2024-09-17T21:47:11.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9209999999999999e-06 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987043+0000", 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1076e-05 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987049+0000", 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000331+0000", 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013281866999999999 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000405+0000", 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4004000000000005e-05 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000491+0000", 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6062000000000004e-05 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1236 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6066485950000002, 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887803000000001, 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1236 2024-09-17T21:47:11.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987058+0000", 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987065+0000", 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1219999999999997e-06 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987070+0000", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9529999999999998e-06 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000520+0000", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013449796999999999 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000592+0000", 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919e-05 2024-09-17T21:47:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000944+0000", 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035145700000000001 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1237 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.606627488, 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049556, 2024-09-17T21:47:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1237 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987080+0000", 2024-09-17T21:47:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987086+0000", 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6309999999999997e-06 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987091+0000", 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000974+0000", 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013882864999999999 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001045+0000", 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0661000000000007e-05 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001126+0000", 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1521999999999996e-05 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1264 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6064565680000005, 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014065061, 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1264 2024-09-17T21:47:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987249+0000", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987252+0000", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987263+0000", 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1289e-05 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987270+0000", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1330000000000001e-06 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001154+0000", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01388413 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001225+0000", 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0975000000000004e-05 2024-09-17T21:47:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001313+0000", 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7404999999999994e-05 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1265 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6064232579999995, 2024-09-17T21:47:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01421443, 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1265 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987282+0000", 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987284+0000", 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.328e-06 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987291+0000", 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987305+0000", 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3344e-05 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001341+0000", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014035903000000001 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001411+0000", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0502999999999999e-05 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001496+0000", 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4383999999999997e-05 2024-09-17T21:47:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1266 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.606391318, 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014670829, 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1266 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987314+0000", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987316+0000", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4059999999999999e-06 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987323+0000", 2024-09-17T21:47:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987334+0000", 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.093e-05 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001524+0000", 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014189715 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001592+0000", 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8132000000000004e-05 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001984+0000", 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039185199999999999 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1267 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6063623480000002, 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014824201, 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1267 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987343+0000", 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987345+0000", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987351+0000", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987356+0000", 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002014+0000", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014657411 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002084+0000", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0596e-05 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002166+0000", 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2083999999999995e-05 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1268 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.6063406990000004, 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014992027999999999, 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1268 2024-09-17T21:47:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987369+0000", 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7129999999999996e-06 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987376+0000", 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987381+0000", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002195+0000", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014814226 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002268+0000", 2024-09-17T21:47:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3279000000000001e-05 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002356+0000", 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7305999999999994e-05 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1269 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.606313535, 2024-09-17T21:47:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015291325999999999, 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1269 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987394+0000", 2024-09-17T21:47:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3989999999999998e-06 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987401+0000", 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8210000000000004e-06 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987406+0000", 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0030000000000002e-06 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002522+0000", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015116368 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002595+0000", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3335000000000002e-05 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002682+0000", 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6927000000000003e-05 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:771 3.1 3:8006aab1:::rbd_data.1112f0bef54a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 8280576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.9895604819999999, 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013777717, 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 771 2024-09-17T21:47:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604145+0000", 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608094+0000", 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039492950000000002 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608114+0000", 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0254e-05 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608122+0000", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608205+0000", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3511999999999996e-05 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608304+0000", 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8524999999999996e-05 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608621+0000", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031733999999999999 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617430+0000", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088086459999999998 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617787+0000", 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035710100000000002 2024-09-17T21:47:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617854+0000", 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7699000000000004e-05 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617922+0000", 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7169999999999996e-05 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:11.697 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:11.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:11.699 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:11.695+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:11.799 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:11.797+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:11.846 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:11.846 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:11.846 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:11.846 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:11.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:11.895 INFO:teuthology.orchestra.run.smithi165.stdout:3046 read 0x33c7000 thru 0x33d34a1 (0xc4a2 bytes) 2024-09-17T21:47:11.895 INFO:teuthology.orchestra.run.smithi165.stdout:3047 read 0x5b4c00 thru 0x5b6163 (0x1564 bytes) 2024-09-17T21:47:11.895 INFO:teuthology.orchestra.run.smithi165.stdout:3048 write 0x5d56c00 thru 0x5d60e96 (0xa297 bytes) 2024-09-17T21:47:11.895 INFO:teuthology.orchestra.run.smithi165.stdout:3051 punch from 0x899400 to 0x8a7600, (0xe200 bytes) 2024-09-17T21:47:11.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:47:11.899 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:11.897+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:11.907 INFO:teuthology.orchestra.run.smithi165.stdout:3053 trunc from 0x851a600 to 0xd781c00 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3054 punch from 0xb53d800 to 0xb543c00, (0x6400 bytes) 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3055 write 0x5098400 thru 0x509cbf6 (0x47f7 bytes) 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3056 punch from 0x26a7200 to 0x26b0e00, (0x9c00 bytes) 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3058 punch from 0x8b5ea00 to 0x8b6de00, (0xf400 bytes) 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3059 write 0x6971000 thru 0x6979be6 (0x8be7 bytes) 2024-09-17T21:47:11.927 INFO:teuthology.orchestra.run.smithi165.stdout:3060 read 0x507a800 thru 0x50853b2 (0xabb3 bytes) 2024-09-17T21:47:11.928 INFO:teuthology.orchestra.run.smithi165.stdout:3062 write 0x2a42a00 thru 0x2a4c20f (0x9810 bytes) 2024-09-17T21:47:11.928 INFO:teuthology.orchestra.run.smithi165.stdout:3063 write 0x93dbc00 thru 0x93e3a04 (0x7e05 bytes) 2024-09-17T21:47:11.928 INFO:teuthology.orchestra.run.smithi165.stdout:3064 write 0xd5e7c00 thru 0xd5efec6 (0x82c7 bytes) 2024-09-17T21:47:11.948 INFO:teuthology.orchestra.run.smithi165.stdout:3068 trunc from 0xd781c00 to 0x11a3800 2024-09-17T21:47:11.967 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:47:11.967 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:11.967 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:47:11.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-09-17T21:47:11.974 INFO:teuthology.orchestra.run.smithi165.stdout:3069 clone 25 order 22 su 2097152 sc 8 2024-09-17T21:47:11.990 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:11.990 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:11.990 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:11.990 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:11.990 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:12.000 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:12.001 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:11.998+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:12.101 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:12.097+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:12.153 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:12.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246200311999999, 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246190011000003, 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:12.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246173937000002, 2024-09-17T21:47:12.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:12.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246162724000001, 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:12.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:12.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:12.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246145286000001, 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:12.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:12.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.246134415999997, 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:12.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:12.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:12.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.245939538999998, 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:12.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.244234869000003, 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:12.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.244224230999997, 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:12.190 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:12.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.244214137999997, 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:12.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:12.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.244204019000001, 2024-09-17T21:47:12.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:12.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:12.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.242073789000003, 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.208 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.242064057999997, 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:12.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.241280699000001, 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:12.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.241265128999999, 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.218 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:12.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.241172227, 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:12.222 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:12.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:12.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.240677607999999, 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:12.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.240533563, 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:12.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:12.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.239848125999998, 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.239485735999999, 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:12.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:12.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:12.244 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:12.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:12.247 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:12.242+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:12.266 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:47:12.264+0000 7f1205775640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:moved 2 / 82 (2.43902%) 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:avg 13.6667 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 1.59861 -> 1.49071 (expected baseline 3.37474) 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.2 with 11 -> 11 pgs (0.804878 -> 0.804878 * mean) 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.3 with 16 -> 15 pgs (1.17073 -> 1.09756 * mean) 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:12.267 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 0.05 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0013 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0014 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 weight 1.0000 -> 0.9500 2024-09-17T21:47:12.268 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T21:47:12.280 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:47:12.280 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-17T21:47:12.366 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:12.364+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:12.376 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:12.376 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:12.376 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:12.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:12.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:12.470 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:12.467+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:12.539 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:12.539 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:12.539 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:12.549 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:12.549 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:12.545+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:12.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:12.553 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:12.571 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:12.572 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:12.588 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:12.589 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:12.606 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:12.606 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:12.619 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:47:12.616+0000 7f1205775640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T21:47:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:moved 7 / 82 (8.53659%) 2024-09-17T21:47:12.619 INFO:teuthology.orchestra.run.smithi007.stdout:avg 13.6667 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 1.59861 -> 2.49444 (expected baseline 3.37474) 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.2 with 11 -> 10 pgs (0.804878 -> 0.731707 * mean) 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.3 with 16 -> 11 pgs (1.17073 -> 0.804878 * mean) 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 1 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0013 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0014 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 weight 1.0000 -> 0.7181 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 weight 1.0000 -> 0.7903 2024-09-17T21:47:12.620 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 weight 1.0000 -> 0.7907 2024-09-17T21:47:12.621 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T21:47:12.623 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:12.623 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:12.633 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:47:12.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-17T21:47:12.641 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:12.642 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:12.649 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:12.647+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:12.663 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:12.663 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:12.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:12.750 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:12.747+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:12.791 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:12.791 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:12.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:12.851 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:12.849+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:12.892 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:47:12.936 INFO:tasks.ceph.mgr.x.smithi053.stderr:2024-09-17T21:47:12.933+0000 7f1205775640 -1 mgr.server reply reply (3) No such process no change 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:moved 7 / 82 (8.53659%) 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:avg 13.6667 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 1.59861 -> 2.49444 (expected baseline 3.37474) 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.2 with 11 -> 10 pgs (0.804878 -> 0.731707 * mean) 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.3 with 16 -> 11 pgs (1.17073 -> 0.804878 * mean) 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 3 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0013 2024-09-17T21:47:12.937 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0014 2024-09-17T21:47:12.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 weight 1.0000 -> 0.7181 2024-09-17T21:47:12.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 weight 1.0000 -> 0.7903 2024-09-17T21:47:12.938 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 weight 1.0000 -> 0.7907 2024-09-17T21:47:12.938 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T21:47:12.949 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-17T21:47:12.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-17T21:47:12.952 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:12.949+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:12.954 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:12.954 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:12.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:12.990 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone24 from 0x11a3800 (overlap 0x421e00) to 0x0 2024-09-17T21:47:13.052 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:13.050+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:13.114 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.115 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:13.167 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:13.165+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:13.236 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.236 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:13.286 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:13.284+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:moved 0 / 82 (0%) 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:avg 13.6667 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 1.59861 -> 1.59861 (expected baseline 3.37474) 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.2 with 11 -> 11 pgs (0.804878 -> 0.804878 * mean) 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.3 with 16 -> 16 pgs (1.17073 -> 1.17073 * mean) 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:oload 1000 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 1 2024-09-17T21:47:13.288 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T21:47:13.289 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0013 2024-09-17T21:47:13.289 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0129 2024-09-17T21:47:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:no change 2024-09-17T21:47:13.301 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-17T21:47:13.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-17T21:47:13.348 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-17T21:47:13.385 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:13.383+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:13.403 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.403 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:13.453 INFO:teuthology.orchestra.run.smithi165.stdout:3072 write 0x3fe4200 thru 0x3fed9fd (0x97fe bytes) 2024-09-17T21:47:13.454 INFO:teuthology.orchestra.run.smithi165.stdout:3073 punch from 0x1c0fa00 to 0x1c19800, (0x9e00 bytes) 2024-09-17T21:47:13.454 INFO:teuthology.orchestra.run.smithi165.stdout:3077 punch from 0xdf0400 to 0xdf4200, (0x3e00 bytes) 2024-09-17T21:47:13.457 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:13.454+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:13.458 INFO:teuthology.orchestra.run.smithi165.stdout:3078 write 0xedc00 thru 0xfc22e (0xe62f bytes) 2024-09-17T21:47:13.458 INFO:teuthology.orchestra.run.smithi165.stdout:3079 read 0x870800 thru 0x878d14 (0x8515 bytes) 2024-09-17T21:47:13.458 INFO:teuthology.orchestra.run.smithi165.stdout:3080 read 0x224c400 thru 0x2258f4f (0xcb50 bytes) 2024-09-17T21:47:13.486 INFO:teuthology.orchestra.run.smithi165.stdout:3081 write 0x6f75000 thru 0x6f7664b (0x164c bytes) 2024-09-17T21:47:13.490 INFO:teuthology.orchestra.run.smithi165.stdout:3084 trunc from 0x6f76800 to 0x70a2400 2024-09-17T21:47:13.498 INFO:teuthology.orchestra.run.smithi165.stdout:3086 write 0x5515800 thru 0x551fc1a (0xa41b bytes) 2024-09-17T21:47:13.503 INFO:teuthology.orchestra.run.smithi165.stdout:truncating to largest ever: 0xedb6e00 2024-09-17T21:47:13.508 INFO:teuthology.orchestra.run.smithi165.stdout:3087 trunc from 0x70a2400 to 0xedb6e00 2024-09-17T21:47:13.522 INFO:teuthology.orchestra.run.smithi165.stdout:3089 write 0xe6de000 thru 0xe6e2ac8 (0x4ac9 bytes) 2024-09-17T21:47:13.523 INFO:teuthology.orchestra.run.smithi165.stdout:3090 read 0xb15ba00 thru 0xb15d3a3 (0x19a4 bytes) 2024-09-17T21:47:13.523 INFO:teuthology.orchestra.run.smithi165.stdout:3092 read 0x6a25c00 thru 0x6a2de2e (0x822f bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3093 read 0x9f8dc00 thru 0x9f9a81c (0xcc1d bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3096 read 0x6f53000 thru 0x6f61bb8 (0xebb9 bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3098 punch from 0xa269e00 to 0xa26de00, (0x4000 bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3101 write 0x9eb7e00 thru 0x9ec72e5 (0xf4e6 bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3102 write 0xa4f8800 thru 0xa4fcae6 (0x42e7 bytes) 2024-09-17T21:47:13.528 INFO:teuthology.orchestra.run.smithi165.stdout:3107 read 0x4414400 thru 0x441d7b4 (0x93b5 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3114 write 0xb43a800 thru 0xb43d964 (0x3165 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3115 read 0xa576000 thru 0xa57b061 (0x5062 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3116 punch from 0x18c0000 to 0x18c2000, (0x2000 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3117 punch from 0xad2b200 to 0xad38800, (0xd600 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3119 read 0xbfad000 thru 0xbfb4678 (0x7679 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3122 read 0x8db8c00 thru 0x8dc4901 (0xbd02 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3123 write 0x5176000 thru 0x517c5f5 (0x65f6 bytes) 2024-09-17T21:47:13.529 INFO:teuthology.orchestra.run.smithi165.stdout:3125 write 0xb212a00 thru 0xb21e7cc (0xbdcd bytes) 2024-09-17T21:47:13.530 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.530 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.530 INFO:teuthology.orchestra.run.smithi165.stdout:3126 trunc from 0xedb6e00 to 0xcb10800 2024-09-17T21:47:13.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:13.545 INFO:teuthology.orchestra.run.smithi165.stdout:3129 read 0x6829400 thru 0x682a9af (0x15b0 bytes) 2024-09-17T21:47:13.548 INFO:teuthology.orchestra.run.smithi165.stdout:3131 punch from 0x50a9600 to 0x50b2e00, (0x9800 bytes) 2024-09-17T21:47:13.548 INFO:teuthology.orchestra.run.smithi165.stdout:3133 trunc from 0xcb10800 to 0xd642e00 2024-09-17T21:47:13.557 INFO:teuthology.orchestra.run.smithi165.stdout:3138 read 0x80b600 thru 0x80dd33 (0x2734 bytes) 2024-09-17T21:47:13.558 INFO:teuthology.orchestra.run.smithi165.stdout:3139 read 0x2e17000 thru 0x2e1e6bb (0x76bc bytes) 2024-09-17T21:47:13.559 INFO:teuthology.orchestra.run.smithi165.stdout:3140 write 0x9f45600 thru 0x9f4769d (0x209e bytes) 2024-09-17T21:47:13.559 INFO:teuthology.orchestra.run.smithi165.stdout:3142 read 0xb0eaa00 thru 0xb0eb416 (0xa17 bytes) 2024-09-17T21:47:13.559 INFO:teuthology.orchestra.run.smithi165.stdout:3144 write 0x6f39000 thru 0x6f41893 (0x8894 bytes) 2024-09-17T21:47:13.560 INFO:teuthology.orchestra.run.smithi165.stdout:3145 read 0x174da00 thru 0x1751e81 (0x4482 bytes) 2024-09-17T21:47:13.560 INFO:teuthology.orchestra.run.smithi165.stdout:3147 read 0x1f8d800 thru 0x1f95e6f (0x8670 bytes) 2024-09-17T21:47:13.561 INFO:teuthology.orchestra.run.smithi165.stdout:3148 read 0x24e5200 thru 0x24ebff6 (0x6df7 bytes) 2024-09-17T21:47:13.561 INFO:teuthology.orchestra.run.smithi165.stdout:3151 read 0x157a000 thru 0x1586f2f (0xcf30 bytes) 2024-09-17T21:47:13.561 INFO:teuthology.orchestra.run.smithi165.stdout:3154 write 0x2e5fe00 thru 0x2e600c2 (0x2c3 bytes) 2024-09-17T21:47:13.562 INFO:teuthology.orchestra.run.smithi165.stdout:3155 write 0x2f3da00 thru 0x2f4c1d0 (0xe7d1 bytes) 2024-09-17T21:47:13.562 INFO:teuthology.orchestra.run.smithi165.stdout:3156 write 0x3f7ac00 thru 0x3f7eab7 (0x3eb8 bytes) 2024-09-17T21:47:13.562 INFO:teuthology.orchestra.run.smithi165.stdout:3157 punch from 0x9b18800 to 0x9b1da00, (0x5200 bytes) 2024-09-17T21:47:13.562 INFO:teuthology.orchestra.run.smithi165.stdout:3158 read 0x19d1200 thru 0x19d74ac (0x62ad bytes) 2024-09-17T21:47:13.573 INFO:teuthology.orchestra.run.smithi165.stdout:3160 write 0xe712400 thru 0xe714fbc (0x2bbd bytes) 2024-09-17T21:47:13.574 INFO:teuthology.orchestra.run.smithi165.stdout:3162 write 0x280e000 thru 0x281a98a (0xc98b bytes) 2024-09-17T21:47:13.574 INFO:teuthology.orchestra.run.smithi165.stdout:3163 write 0x92bb400 thru 0x92c90c9 (0xdcca bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3167 punch from 0xe618000 to 0xe61ca00, (0x4a00 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3168 punch from 0x55d800 to 0x561400, (0x3c00 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3169 read 0x64bc000 thru 0x64c7305 (0xb306 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3170 read 0x8c24e00 thru 0x8c25f38 (0x1139 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3173 punch from 0x8908e00 to 0x8909e00, (0x1000 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3175 read 0xa1d0e00 thru 0xa1dba68 (0xac69 bytes) 2024-09-17T21:47:13.579 INFO:teuthology.orchestra.run.smithi165.stdout:3176 trunc from 0xe715000 to 0xc203a00 2024-09-17T21:47:13.582 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:13.579+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:13.595 INFO:teuthology.orchestra.run.smithi165.stdout:3179 trunc from 0xc203a00 to 0xa5b0a00 2024-09-17T21:47:13.608 INFO:teuthology.orchestra.run.smithi165.stdout:3180 punch from 0x6646200 to 0x6650e00, (0xac00 bytes) 2024-09-17T21:47:13.613 INFO:teuthology.orchestra.run.smithi165.stdout:3182 trunc from 0xa5b0a00 to 0xd672c00 2024-09-17T21:47:13.622 INFO:teuthology.orchestra.run.smithi165.stdout:3183 punch from 0xd63a00 to 0xd6fc00, (0xc200 bytes) 2024-09-17T21:47:13.630 INFO:teuthology.orchestra.run.smithi165.stdout:3186 write 0x2402600 thru 0x2409dfd (0x77fe bytes) 2024-09-17T21:47:13.630 INFO:teuthology.orchestra.run.smithi165.stdout:3188 punch from 0xd0b1800 to 0xd0bf000, (0xd800 bytes) 2024-09-17T21:47:13.630 INFO:teuthology.orchestra.run.smithi165.stdout:3190 write 0x943e400 thru 0x9448a3a (0xa63b bytes) 2024-09-17T21:47:13.631 INFO:teuthology.orchestra.run.smithi165.stdout:3191 write 0x4b4b200 thru 0x4b4d8df (0x26e0 bytes) 2024-09-17T21:47:13.631 INFO:teuthology.orchestra.run.smithi165.stdout:3192 read 0xb500a00 thru 0xb50edcd (0xe3ce bytes) 2024-09-17T21:47:13.644 INFO:teuthology.orchestra.run.smithi165.stdout:3194 trunc from 0xd672c00 to 0x1eac200 2024-09-17T21:47:13.657 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:13.654+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:13.678 INFO:teuthology.orchestra.run.smithi165.stdout:3196 trunc from 0x1eac200 to 0xb36bc00 2024-09-17T21:47:13.687 INFO:teuthology.orchestra.run.smithi165.stdout:3197 read 0xa14f400 thru 0xa15eac5 (0xf6c6 bytes) 2024-09-17T21:47:13.687 INFO:teuthology.orchestra.run.smithi165.stdout:3198 write 0x9449000 thru 0x944ca81 (0x3a82 bytes) 2024-09-17T21:47:13.687 INFO:teuthology.orchestra.run.smithi165.stdout:3200 punch from 0x3279a00 to 0x3289800, (0xfe00 bytes) 2024-09-17T21:47:13.697 INFO:teuthology.orchestra.run.smithi165.stdout:3201 trunc from 0xb36bc00 to 0x2b47e00 2024-09-17T21:47:13.702 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.702 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:13.727 INFO:teuthology.orchestra.run.smithi165.stdout:3202 write 0x853e000 thru 0x854856b (0xa56c bytes) 2024-09-17T21:47:13.734 INFO:teuthology.orchestra.run.smithi165.stdout:3206 trunc from 0x8548600 to 0x2aeb600 2024-09-17T21:47:13.752 INFO:teuthology.orchestra.run.smithi165.stdout:3207 read 0x1e5b400 thru 0x1e6af4b (0xfb4c bytes) 2024-09-17T21:47:13.758 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:13.755+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:13.759 INFO:teuthology.orchestra.run.smithi165.stdout:3208 read 0x1244000 thru 0x1252b91 (0xeb92 bytes) 2024-09-17T21:47:13.759 INFO:teuthology.orchestra.run.smithi165.stdout:3210 read 0x4d7200 thru 0x4e007c (0x8e7d bytes) 2024-09-17T21:47:13.764 INFO:teuthology.orchestra.run.smithi165.stdout:3211 trunc from 0x2aeb600 to 0x64a0000 2024-09-17T21:47:13.773 INFO:teuthology.orchestra.run.smithi165.stdout:3213 read 0x2ffbe00 thru 0x2ffcd8e (0xf8f bytes) 2024-09-17T21:47:13.781 INFO:teuthology.orchestra.run.smithi165.stdout:3215 read 0xb80000 thru 0xb80fa2 (0xfa3 bytes) 2024-09-17T21:47:13.781 INFO:teuthology.orchestra.run.smithi165.stdout:3217 write 0x4e8f000 thru 0x4e9c36e (0xd36f bytes) 2024-09-17T21:47:13.785 INFO:teuthology.orchestra.run.smithi165.stdout:3218 trunc from 0x64a0000 to 0xa3faa00 2024-09-17T21:47:13.798 INFO:teuthology.orchestra.run.smithi165.stdout:3222 trunc from 0xa3faa00 to 0xa779800 2024-09-17T21:47:13.826 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.826 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.830 INFO:teuthology.orchestra.run.smithi165.stdout:3224 write 0xed89800 thru 0xed927bf (0x8fc0 bytes) 2024-09-17T21:47:13.831 INFO:teuthology.orchestra.run.smithi165.stdout:3226 punch from 0xa443000 to 0xa443c00, (0xc00 bytes) 2024-09-17T21:47:13.832 INFO:teuthology.orchestra.run.smithi165.stdout:3228 write 0x2c5e00 thru 0x2c9e98 (0x4099 bytes) 2024-09-17T21:47:13.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:13.845 INFO:teuthology.orchestra.run.smithi165.stdout:3229 trunc from 0xed92800 to 0x6aafe00 2024-09-17T21:47:13.859 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:13.856+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:13.868 INFO:teuthology.orchestra.run.smithi165.stdout:3231 read 0x2ed2400 thru 0x2ed24b7 (0xb8 bytes) 2024-09-17T21:47:13.891 INFO:teuthology.orchestra.run.smithi165.stdout:3234 write 0xd065800 thru 0xd06d89f (0x80a0 bytes) 2024-09-17T21:47:13.895 INFO:teuthology.orchestra.run.smithi007.stdout:moved 2 / 82 (2.43902%) 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:avg 13.6667 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:stddev 1.59861 -> 1.49071 (expected baseline 3.37474) 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:min osd.2 with 11 -> 11 pgs (0.804878 -> 0.804878 * mean) 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:max osd.3 with 16 -> 15 pgs (1.17073 -> 1.09756 * mean) 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:oload 110 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:max_change 0.05 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:max_change_osds 4 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:average_utilization 0.0013 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:overload_utilization 0.0014 2024-09-17T21:47:13.896 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-17T21:47:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-17T21:47:13.897 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 weight 1.0000 -> 0.9500 2024-09-17T21:47:13.899 INFO:teuthology.orchestra.run.smithi165.stdout:3236 read 0x4232c00 thru 0x42346e7 (0x1ae8 bytes) 2024-09-17T21:47:13.899 INFO:teuthology.orchestra.run.smithi165.stdout:3238 write 0x51bee00 thru 0x51c87dc (0x99dd bytes) 2024-09-17T21:47:13.899 INFO:teuthology.orchestra.run.smithi165.stdout:3245 punch from 0xbd28200 to 0xbd35600, (0xd400 bytes) 2024-09-17T21:47:13.901 INFO:teuthology.orchestra.run.smithi165.stdout:3246 write 0x9f94400 thru 0x9f9f2d7 (0xaed8 bytes) 2024-09-17T21:47:13.901 INFO:teuthology.orchestra.run.smithi165.stdout:3247 write 0xc490e00 thru 0xc495f26 (0x5127 bytes) 2024-09-17T21:47:13.902 INFO:teuthology.orchestra.run.smithi165.stdout:3248 read 0x78eb200 thru 0x78f3ac6 (0x88c7 bytes) 2024-09-17T21:47:13.902 INFO:teuthology.orchestra.run.smithi165.stdout:3249 write 0xc69b000 thru 0xc6a7729 (0xc72a bytes) 2024-09-17T21:47:13.902 INFO:teuthology.orchestra.run.smithi165.stdout:3250 write 0xf4f400 thru 0xf58580 (0x9181 bytes) 2024-09-17T21:47:13.902 INFO:teuthology.orchestra.run.smithi165.stdout:3251 read 0xcac0000 thru 0xcac5a4a (0x5a4b bytes) 2024-09-17T21:47:13.903 INFO:teuthology.orchestra.run.smithi165.stdout:3253 punch from 0x28e1e00 to 0x28ea400, (0x8600 bytes) 2024-09-17T21:47:13.903 INFO:teuthology.orchestra.run.smithi165.stdout:3254 punch from 0x37e7600 to 0x37ef600, (0x8000 bytes) 2024-09-17T21:47:13.909 INFO:teuthology.orchestra.run.smithi165.stdout:3256 trunc from 0xd06da00 to 0x7911800 2024-09-17T21:47:13.954 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:13.954 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:13.960 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:13.958+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:13.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:14.061 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:14.058+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:14.070 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.070 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:14.161 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:14.159+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:14.186 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.186 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.195 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:14.262 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:14.259+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:14.342 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.343 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:14.362 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:14.360+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:14.459 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.459 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.463 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:14.461+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:14.467 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:14.565 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:14.562+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:14.574 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.574 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:14.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:14.690 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.690 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.691 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:14.688+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:14.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:14.785 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:14.783+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:14.849 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:14.849 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:14.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:14.898 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:14.896+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:14.908 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:47:14.968 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:14.965+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:14.994 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:15.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:15.024 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:15.024 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:15.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:15.040 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:15.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:15.068 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:15.066+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:15.092 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:15.093 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:15.110 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:15.111 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.335390214, 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:47:15.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:47:15.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.325739249999998, 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:47:15.155 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:47:15.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.241793381000001, 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:47:15.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:47:15.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.240331245999997, 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:15.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:47:15.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.237720127999999, 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:47:15.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:47:15.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.173605670999997, 2024-09-17T21:47:15.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:47:15.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.172877448999998, 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:47:15.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:47:15.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.839300313000003, 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:47:15.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:47:15.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:47:15.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.272541811000004, 2024-09-17T21:47:15.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:15.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.265019668999997, 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:47:15.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1128 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1547305970000004, 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099832400000000009, 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1128 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:15.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990586+0000", 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.125e-06 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990593+0000", 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6919999999999996e-06 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990598+0000", 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9830000000000004e-06 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000414+0000", 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098157309999999994 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000481+0000", 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7303000000000003e-05 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000567+0000", 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5923999999999999e-05 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1129 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1547100459999999, 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010143592999999999, 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1129 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990605+0000", 2024-09-17T21:47:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990607+0000", 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990613+0000", 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990618+0000", 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000596+0000", 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099777809999999998 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000662+0000", 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5891999999999999e-05 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000748+0000", 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6372999999999994e-05 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1224 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1478606130000006, 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010061038, 2024-09-17T21:47:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1224 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997457+0000", 2024-09-17T21:47:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3499999999999999e-06 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998930+0000", 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014735079999999999 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998947+0000", 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6977999999999999e-05 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007338+0000", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083907679999999998 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007417+0000", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9008e-05 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007515+0000", 2024-09-17T21:47:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7958e-05 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1255 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1458847009999999, 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010112939, 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1255 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999432+0000", 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249999999999999e-06 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999439+0000", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999445+0000", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2730000000000002e-06 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009368+0000", 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099234240000000001 2024-09-17T21:47:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009446+0000", 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8209999999999998e-05 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009543+0000", 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6353000000000004e-05 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1256 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1458632360000003, 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010198560000000001, 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1256 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999452+0000", 2024-09-17T21:47:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999453+0000", 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.945e-06 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999460+0000", 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7789999999999998e-06 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999469+0000", 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6309999999999992e-06 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009556+0000", 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010086927000000001 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009597+0000", 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1038999999999998e-05 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009650+0000", 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2831999999999998e-05 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1288 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.142846917, 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010050948000000001, 2024-09-17T21:47:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1288 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002468+0000", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002470+0000", 2024-09-17T21:47:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.384e-06 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002477+0000", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002482+0000", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1089999999999997e-06 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012369+0000", 2024-09-17T21:47:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098866030000000008 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012434+0000", 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5512999999999994e-05 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012518+0000", 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4089000000000001e-05 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1307 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1348296649999998, 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099939750000000004, 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1307 2024-09-17T21:47:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010488+0000", 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0240000000000002e-06 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010495+0000", 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010508+0000", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3410999999999999e-05 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017483+0000", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069745320000000003 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017570+0000", 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7057000000000004e-05 2024-09-17T21:47:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017970+0000", 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040009600000000001 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020303+0000", 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023332729999999999 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020439+0000", 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135783 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020462+0000", 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2469000000000001e-05 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020479+0000", 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7144999999999999e-05 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1309 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1347972679999998, 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010826840000000001, 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1309 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010518+0000", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010521+0000", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.207e-06 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010528+0000", 2024-09-17T21:47:15.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010533+0000", 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1930000000000001e-06 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018747+0000", 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082139880000000002 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018835+0000", 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7560999999999997e-05 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.019193+0000", 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035790399999999998 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020506+0000", 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013139099999999999 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021255+0000", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074901999999999998 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021305+0000", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230999999999997e-05 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021344+0000", 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9252000000000003e-05 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1311 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1347737739999992, 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012406011, 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1311 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010544+0000", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.977e-06 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010551+0000", 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7240000000000003e-06 2024-09-17T21:47:15.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010556+0000", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0989999999999998e-06 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020524+0000", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.009968457 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020568+0000", 2024-09-17T21:47:15.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4132999999999999e-05 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020784+0000", 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021525299999999999 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022838+0000", 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020542030000000001 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022848+0000", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9879999999999999e-06 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022894+0000", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5803000000000002e-05 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022947+0000", 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2896999999999999e-05 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1312 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1323912210000007, 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011273178999999999, 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:15.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1312 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012924+0000", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012937+0000", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2662e-05 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012950+0000", 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-17T21:47:15.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012958+0000", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9140000000000008e-06 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021388+0000", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084298759999999993 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021609+0000", 2024-09-17T21:47:15.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221256 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021953+0000", 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034352200000000002 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022955+0000", 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010025909999999999 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024087+0000", 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011314109999999999 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024148+0000", 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1001000000000001e-05 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024196+0000", 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8474999999999999e-05 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:15.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:15.261 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:15.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:15.263 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:15.260+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:15.363 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:15.360+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:15.382 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:15.382 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:15.383 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:15.383 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:15.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:15.476 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:15.474+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:15.541 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:15.541 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:15.541 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:15.541 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:15.541 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:15.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:15.591 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:15.589+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:15.666 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:15.664+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:15.699 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:15.699 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:15.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.537086353000007, 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:47:15.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:47:15.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.876101063999997, 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:47:15.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.858529673, 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:47:15.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.858495980000001, 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:47:15.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855654676999997, 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:47:15.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:47:15.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855577365999999, 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855554972999997, 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.734 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855533852000001, 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:15.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:47:15.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855513107999997, 2024-09-17T21:47:15.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:15.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:47:15.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855098464000001, 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:47:15.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.855063272999999, 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:47:15.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.753 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.854171809999997, 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:47:15.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.853805741000002, 2024-09-17T21:47:15.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.800498523000002, 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.763 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:47:15.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.767 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.800462750999998, 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.768 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:47:15.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.796324378999998, 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:47:15.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:47:15.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.791130658, 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.779 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:47:15.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.398624065, 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:47:15.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:47:15.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:47:15.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.827324332000003, 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:47:15.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:47:15.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.821776968999998, 2024-09-17T21:47:15.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:15.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:47:15.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.803 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:15.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:15.805 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:15.802+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:15.904 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:47:15.905 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:15.903+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:15.934 INFO:teuthology.orchestra.run.smithi165.stdout:3258 read 0x66d2400 thru 0x66d4881 (0x2482 bytes) 2024-09-17T21:47:15.935 INFO:teuthology.orchestra.run.smithi165.stdout:3259 punch from 0x72ff400 to 0x7302e00, (0x3a00 bytes) 2024-09-17T21:47:15.941 INFO:teuthology.orchestra.run.smithi165.stdout:3260 trunc from 0x7911800 to 0x8d1ea00 2024-09-17T21:47:15.952 INFO:teuthology.orchestra.run.smithi165.stdout:3262 punch from 0xf5a000 to 0xf5c200, (0x2200 bytes) 2024-09-17T21:47:15.957 INFO:teuthology.orchestra.run.smithi165.stdout:3263 read 0x357aa00 thru 0x3587dfb (0xd3fc bytes) 2024-09-17T21:47:15.957 INFO:teuthology.orchestra.run.smithi165.stdout:3264 write 0x1103200 thru 0x1105e6e (0x2c6f bytes) 2024-09-17T21:47:15.960 INFO:teuthology.orchestra.run.smithi165.stdout:3266 trunc from 0x8d1ea00 to 0xa58cc00 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5299.0:1358 3.9 3:96095e16:::rbd_header.11122b976e41:head [call rbd.get_size in=19b,call rbd.get_features in=24b,call rbd.get_flags in=20b,call rbd.get_snapcontext in=18b,call lock.get_info in=30b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:15.966852+0000", 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00015823700000000001, 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019436599999999999, 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:15.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5299", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3432158305", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1358 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966852+0000", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966852+0000", 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966852+0000", 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5899999999999996e-07 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966856+0000", 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1779999999999996e-06 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966866+0000", 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000008e-06 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966871+0000", 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.096e-06 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966914+0000", 2024-09-17T21:47:15.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3503e-05 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967011+0000", 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6637000000000003e-05 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:12616 3.9 3.687a9069 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:15.966979+0000", 2024-09-17T21:47:15.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.0670999999999998e-05, 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000136787, 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 12616 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966979+0000", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966979+0000", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966980+0000", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1299999999999999e-07 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966985+0000", 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7520000000000004e-06 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.966994+0000", 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6369999999999994e-06 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967001+0000", 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4289999999999998e-06 2024-09-17T21:47:15.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5299.0:1359 3.9 3.687a9069 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:15.967029+0000", 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999804, 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014538600000000001, 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5299", 2024-09-17T21:47:15.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3432158305", 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1359 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967029+0000", 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967029+0000", 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:15.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967030+0000", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6199999999999997e-07 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967034+0000", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967040+0000", 2024-09-17T21:47:15.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0000000000000002e-06 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:15.967045+0000", 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5990000000000004e-06 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 3 2024-09-17T21:47:15.980 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:15.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:3267 write 0xec8fc00 thru 0xec912a8 (0x16a9 bytes) 2024-09-17T21:47:15.992 INFO:teuthology.orchestra.run.smithi165.stdout:3270 write 0xea25e00 thru 0xea33ccd (0xdece bytes) 2024-09-17T21:47:15.992 INFO:teuthology.orchestra.run.smithi165.stdout:3271 punch from 0xc00c800 to 0xc019600, (0xce00 bytes) 2024-09-17T21:47:15.995 INFO:teuthology.orchestra.run.smithi165.stdout:3272 read 0xccd6000 thru 0xccd8ad6 (0x2ad7 bytes) 2024-09-17T21:47:15.996 INFO:teuthology.orchestra.run.smithi165.stdout:3276 write 0x754000 thru 0x756719 (0x271a bytes) 2024-09-17T21:47:15.996 INFO:teuthology.orchestra.run.smithi165.stdout:3277 read 0x34f3a00 thru 0x35006e5 (0xcce6 bytes) 2024-09-17T21:47:15.996 INFO:teuthology.orchestra.run.smithi165.stdout:3278 write 0x800f000 thru 0x8012a9b (0x3a9c bytes) 2024-09-17T21:47:16.006 INFO:teuthology.orchestra.run.smithi165.stdout:3279 trunc from 0xec91400 to 0x7e8c400 2024-09-17T21:47:16.025 INFO:teuthology.orchestra.run.smithi165.stdout:3282 read 0x9eae00 thru 0x9eb9ef (0xbf0 bytes) 2024-09-17T21:47:16.025 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:16.023+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:16.027 INFO:teuthology.orchestra.run.smithi165.stdout:3283 read 0x78f0a00 thru 0x78fc876 (0xbe77 bytes) 2024-09-17T21:47:16.027 INFO:teuthology.orchestra.run.smithi165.stdout:3285 read 0x52e5e00 thru 0x52e6c27 (0xe28 bytes) 2024-09-17T21:47:16.027 INFO:teuthology.orchestra.run.smithi165.stdout:3288 read 0x650e600 thru 0x6517112 (0x8b13 bytes) 2024-09-17T21:47:16.027 INFO:teuthology.orchestra.run.smithi165.stdout:3289 punch from 0x553c000 to 0x5547c00, (0xbc00 bytes) 2024-09-17T21:47:16.035 INFO:teuthology.orchestra.run.smithi165.stdout:3291 trunc from 0x7e8c400 to 0xbda7400 2024-09-17T21:47:16.043 INFO:teuthology.orchestra.run.smithi165.stdout:3294 read 0x9d21e00 thru 0x9d23184 (0x1385 bytes) 2024-09-17T21:47:16.047 INFO:teuthology.orchestra.run.smithi165.stdout:3296 trunc from 0xbda7400 to 0x91b3400 2024-09-17T21:47:16.058 INFO:teuthology.orchestra.run.smithi165.stdout:3297 read 0x126d400 thru 0x1278f70 (0xbb71 bytes) 2024-09-17T21:47:16.063 INFO:teuthology.orchestra.run.smithi165.stdout:3298 write 0x6ef2c00 thru 0x6ef8985 (0x5d86 bytes) 2024-09-17T21:47:16.063 INFO:teuthology.orchestra.run.smithi165.stdout:3299 read 0x1822e00 thru 0x182fdf0 (0xcff1 bytes) 2024-09-17T21:47:16.063 INFO:teuthology.orchestra.run.smithi165.stdout:3300 write 0x5967800 thru 0x5968d12 (0x1513 bytes) 2024-09-17T21:47:16.064 INFO:teuthology.orchestra.run.smithi165.stdout:3302 write 0x856ea00 thru 0x857acb1 (0xc2b2 bytes) 2024-09-17T21:47:16.074 INFO:teuthology.orchestra.run.smithi165.stdout:3304 trunc from 0x91b3400 to 0x25ac00 2024-09-17T21:47:16.107 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:16.105+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:16.110 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:16.110 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:16.111 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:16.111 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:16.111 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:16.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:16.126 INFO:teuthology.orchestra.run.smithi165.stdout:3309 write 0xde94600 thru 0xde970ef (0x2af0 bytes) 2024-09-17T21:47:16.128 INFO:teuthology.orchestra.run.smithi165.stdout:3310 write 0x45e5a00 thru 0x45f15ea (0xbbeb bytes) 2024-09-17T21:47:16.128 INFO:teuthology.orchestra.run.smithi165.stdout:3312 read 0x8cbde00 thru 0x8cc74df (0x96e0 bytes) 2024-09-17T21:47:16.128 INFO:teuthology.orchestra.run.smithi165.stdout:3313 clone 26 order 20 su 16384 sc 11 2024-09-17T21:47:16.208 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:16.205+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:16.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:16.269 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.314845233, 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:16.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:16.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.312820180999999, 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:16.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:16.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:16.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.298834124999999, 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:16.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.296872917999998, 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:16.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:16.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.294839561000003, 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:16.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.292861643000002, 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:16.292 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:16.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:16.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.292838848999999, 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:16.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.286929786000002, 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:16.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.284898726999998, 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:16.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:16.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.284872153000002, 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:16.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.28426236, 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.316 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:16.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:16.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.284244774000001, 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:16.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.284229817000003, 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:16.328 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:16.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:16.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:16.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:16.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.568548329000002, 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:16.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:16.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.568517708000002, 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:16.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.568493089999997, 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:16.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:16.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.568471142999996, 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:16.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:16.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:16.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:16.359 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.568449673000003, 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:16.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:16.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:16.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.527032869999999, 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:16.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:16.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.525988409999997, 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.376 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:16.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:16.378 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:16.375+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:16.478 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:16.476+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:16.526 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:16.526 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:16.526 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:16.526 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:16.534 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:16.579 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:16.577+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:16.641 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:16.641 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:16.641 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:16.641 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:16.641 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:16.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:16.690 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:16.688+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:16.782 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:16.779+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:16.798 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:16.798 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824426561999999, 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:16.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:16.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:16.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824404069000003, 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824379809, 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:16.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:16.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:16.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:16.811 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824353926000001, 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.812 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:16.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:16.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824328925000003, 2024-09-17T21:47:16.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:16.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:16.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824289125999996, 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:16.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:16.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824273380999998, 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:16.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824261845999999, 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.824251545999999, 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:16.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822280302999999, 2024-09-17T21:47:16.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:16.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:16.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822267154999999, 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822256549000002, 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:16.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:16.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822246211, 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:16.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822236203999999, 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:16.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822226252999997, 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:16.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:16.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.822210945000002, 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:16.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:16.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:16.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:16.869 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.820235058000002, 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.820224101999997, 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:16.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:16.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.820211612999998, 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.882 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.820201150000003, 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:16.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:16.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:16.887 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:16.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:16.889 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:16.886+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:16.989 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:16.987+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12791 3.10 3.df7d7b50 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e87)", 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:16.971658+0000", 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.038458383999999998, 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.038481145000000001, 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:17.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971658+0000", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:17.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971658+0000", 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971661+0000", 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2600000000000001e-06 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971676+0000", 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5449000000000001e-05 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971712+0000", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5781999999999997e-05 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971725+0000", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3298999999999999e-05 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971822+0000", 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6436999999999998e-05 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:17.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:16.971832+0000", 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.061e-05 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:47:17.016 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:17.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:17.089 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:17.087+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:17.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:17.128 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:17.128 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:17.128 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:17.128 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:17.128 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:17.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:17.221 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:17.219+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:17.246 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:17.264 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:17.280 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:17.281 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:17.286 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.287 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.291 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:17.288+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:17.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:17.297 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:17.298 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:17.315 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:17.315 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:17.333 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:17.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:17.392 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:17.390+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.405 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:17.493 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:17.491+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:17.521 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.521 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.530 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:17.595 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:17.593+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:17.637 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.638 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:17.696 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:17.693+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:17.754 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.754 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:17.797 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:17.795+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:17.869 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.869 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:17.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:17.898 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:17.896+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:17.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:47:17.958 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone25 intact 2024-09-17T21:47:17.991 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:17.991 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.000 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:18.000 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:17.997+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:18.100 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:18.097+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:18.115 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.115 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:18.208 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:18.206+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:18.274 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.274 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.283 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:18.302 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:18.300+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:18.402 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-17T21:47:18.403 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:18.401+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:18.432 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.432 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:18.460 INFO:teuthology.orchestra.run.smithi165.stdout:3314 read 0x22b3600 thru 0x22bf75d (0xc15e bytes) 2024-09-17T21:47:18.461 INFO:teuthology.orchestra.run.smithi165.stdout:3317 punch from 0x4259200 to 0x425c000, (0x2e00 bytes) 2024-09-17T21:47:18.474 INFO:teuthology.orchestra.run.smithi165.stdout:3318 write 0xa3abc00 thru 0xa3b3f0f (0x8310 bytes) 2024-09-17T21:47:18.477 INFO:teuthology.orchestra.run.smithi165.stdout:3321 trunc from 0xde97200 to 0xbd0aa00 2024-09-17T21:47:18.504 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:18.502+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:18.510 INFO:teuthology.orchestra.run.smithi165.stdout:3324 trunc from 0xbd0aa00 to 0x6609000 2024-09-17T21:47:18.548 INFO:teuthology.orchestra.run.smithi165.stdout:3326 write 0x22de00 thru 0x239a09 (0xbc0a bytes) 2024-09-17T21:47:18.548 INFO:teuthology.orchestra.run.smithi165.stdout:3328 punch from 0x27ece00 to 0x27f8400, (0xb600 bytes) 2024-09-17T21:47:18.557 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.557 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.568 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:18.579 INFO:teuthology.orchestra.run.smithi165.stdout:3330 write 0xb03ce00 thru 0xb0444f0 (0x76f1 bytes) 2024-09-17T21:47:18.579 INFO:teuthology.orchestra.run.smithi165.stdout:3331 punch from 0x6bdd000 to 0x6be9200, (0xc200 bytes) 2024-09-17T21:47:18.585 INFO:teuthology.orchestra.run.smithi165.stdout:3332 trunc from 0xb044600 to 0x714b800 2024-09-17T21:47:18.603 INFO:teuthology.orchestra.run.smithi165.stdout:3333 trunc from 0x714b800 to 0x87b3200 2024-09-17T21:47:18.604 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:18.602+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:18.612 INFO:teuthology.orchestra.run.smithi165.stdout:3341 read 0x5d46800 thru 0x5d49f18 (0x3719 bytes) 2024-09-17T21:47:18.613 INFO:teuthology.orchestra.run.smithi165.stdout:3342 read 0x65b9000 thru 0x65c2628 (0x9629 bytes) 2024-09-17T21:47:18.618 INFO:teuthology.orchestra.run.smithi165.stdout:3343 read 0xeb9600 thru 0xec0dcb (0x77cc bytes) 2024-09-17T21:47:18.618 INFO:teuthology.orchestra.run.smithi165.stdout:3345 read 0x1158c00 thru 0x1168974 (0xfd75 bytes) 2024-09-17T21:47:18.619 INFO:teuthology.orchestra.run.smithi165.stdout:3346 read 0x5f72800 thru 0x5f7da77 (0xb278 bytes) 2024-09-17T21:47:18.619 INFO:teuthology.orchestra.run.smithi165.stdout:3347 read 0x25b2a00 thru 0x25b5797 (0x2d98 bytes) 2024-09-17T21:47:18.619 INFO:teuthology.orchestra.run.smithi165.stdout:3348 punch from 0x3356400 to 0x3358000, (0x1c00 bytes) 2024-09-17T21:47:18.642 INFO:teuthology.orchestra.run.smithi165.stdout:3349 write 0xea12800 thru 0xea19c42 (0x7443 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3350 read 0xb0f1000 thru 0xb0f71d4 (0x61d5 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3351 read 0x24e6000 thru 0x24ebfe9 (0x5fea bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3352 read 0x7d67200 thru 0x7d72ba4 (0xb9a5 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3353 write 0x5bfc000 thru 0x5c0b905 (0xf906 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3356 read 0x8802000 thru 0x880e1af (0xc1b0 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3357 read 0x5381c00 thru 0x5389cd4 (0x80d5 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3359 punch from 0x9c8a800 to 0x9c8fe00, (0x5600 bytes) 2024-09-17T21:47:18.643 INFO:teuthology.orchestra.run.smithi165.stdout:3360 punch from 0x524e600 to 0x5255000, (0x6a00 bytes) 2024-09-17T21:47:18.647 INFO:teuthology.orchestra.run.smithi165.stdout:3361 trunc from 0xea19e00 to 0xb77c200 2024-09-17T21:47:18.676 INFO:teuthology.orchestra.run.smithi165.stdout:3363 write 0x1aabc00 thru 0x1abb6b7 (0xfab8 bytes) 2024-09-17T21:47:18.677 INFO:teuthology.orchestra.run.smithi165.stdout:3364 read 0x1f0fc00 thru 0x1f192a8 (0x96a9 bytes) 2024-09-17T21:47:18.680 INFO:teuthology.orchestra.run.smithi165.stdout:3366 write 0x1977a00 thru 0x1984659 (0xcc5a bytes) 2024-09-17T21:47:18.680 INFO:teuthology.orchestra.run.smithi165.stdout:3371 write 0x9889a00 thru 0x98969f1 (0xcff2 bytes) 2024-09-17T21:47:18.680 INFO:teuthology.orchestra.run.smithi165.stdout:3372 punch from 0x825a800 to 0x8269800, (0xf000 bytes) 2024-09-17T21:47:18.681 INFO:teuthology.orchestra.run.smithi165.stdout:3375 read 0x5232200 thru 0x523e646 (0xc447 bytes) 2024-09-17T21:47:18.681 INFO:teuthology.orchestra.run.smithi165.stdout:3377 write 0x4150000 thru 0x4150665 (0x666 bytes) 2024-09-17T21:47:18.681 INFO:teuthology.orchestra.run.smithi165.stdout:3378 write 0xb070200 thru 0xb07bcf0 (0xbaf1 bytes) 2024-09-17T21:47:18.681 INFO:teuthology.orchestra.run.smithi165.stdout:3379 write 0x27f8000 thru 0x27fed62 (0x6d63 bytes) 2024-09-17T21:47:18.690 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.690 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:18.705 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:18.703+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:18.715 INFO:teuthology.orchestra.run.smithi165.stdout:3383 write 0xe039200 thru 0xe0471be (0xdfbf bytes) 2024-09-17T21:47:18.715 INFO:teuthology.orchestra.run.smithi165.stdout:3385 write 0xbcb8a00 thru 0xbcb9b72 (0x1173 bytes) 2024-09-17T21:47:18.715 INFO:teuthology.orchestra.run.smithi165.stdout:3387 read 0x7ea7000 thru 0x7eb1b63 (0xab64 bytes) 2024-09-17T21:47:18.726 INFO:teuthology.orchestra.run.smithi165.stdout:3388 trunc from 0xe047200 to 0x417a200 2024-09-17T21:47:18.775 INFO:teuthology.orchestra.run.smithi165.stdout:3389 punch from 0x3aede00 to 0x3af3600, (0x5800 bytes) 2024-09-17T21:47:18.800 INFO:teuthology.orchestra.run.smithi165.stdout:3391 write 0xab79600 thru 0xab82551 (0x8f52 bytes) 2024-09-17T21:47:18.806 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:18.804+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:18.811 INFO:teuthology.orchestra.run.smithi165.stdout:3395 trunc from 0xab82600 to 0x10e4600 2024-09-17T21:47:18.817 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.817 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.829 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:18.889 INFO:teuthology.orchestra.run.smithi165.stdout:3400 write 0x646ca00 thru 0x646fd5f (0x3360 bytes) 2024-09-17T21:47:18.903 INFO:teuthology.orchestra.run.smithi165.stdout:3402 write 0x74d3600 thru 0x74d3fd1 (0x9d2 bytes) 2024-09-17T21:47:18.908 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:18.905+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:18.912 INFO:teuthology.orchestra.run.smithi165.stdout:3405 trunc from 0x74d4000 to 0x6d02800 2024-09-17T21:47:18.912 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:47:18.912 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:18.912 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:47:18.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:47:18.917 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:47:18.943 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:18.943 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:18.943 INFO:teuthology.orchestra.run.smithi165.stdout:3407 trunc from 0x6d02800 to 0x8e8200 2024-09-17T21:47:18.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:18.988 INFO:teuthology.orchestra.run.smithi165.stdout:3408 clone 27 order 21 su 65536 sc 8 2024-09-17T21:47:19.009 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:19.006+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:19.065 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:19.065 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:19.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:19.110 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:19.107+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:19.211 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:19.208+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:19.220 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:19.220 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:19.224 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 89, 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:47:18.911889+0000", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:45:45.117729+0000", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:47:06.858395+0000", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:47:19.225 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-17T21:47:19.226 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:19.227 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:47:19.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:19.229 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:19.230 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:19.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:47:19.232 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T21:47:19.233 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:19.234 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:47:19.235 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:19.236 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:19.237 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:19.238 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:19.239 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.25, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.25, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:47:19.240 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:19.241 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.242 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "87", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 28, 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 87, 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:19.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:19.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:19.245 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.309999942779541, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.309999942779541, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.309999942779541, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.309999942779541, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:19.247 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 87, 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:47:19.249 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:47:19.250 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:47:19.251 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.253 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.254 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.255 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:19.258 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 80, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.259 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.260 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.261 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:47:19.264 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.265 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.267 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/2260220328", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/2260220328", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/2260220328", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/2260220328", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.270 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 87, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.273 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.274 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:47:19.276 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:47:19.281 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.282 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:19.284 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.285 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.286 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:47:19.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.288 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:19.291 INFO:tasks.thrashosds.thrasher:j is {'epoch': 89, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:47:18.911889+0000', 'last_up_change': '2024-09-17T21:45:45.117729+0000', 'last_in_change': '2024-09-17T21:47:06.858395+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 28, 'snap_epoch': 87, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.309999942779541, 'score_stable': 1.309999942779541, 'optimal_score': 1, 'raw_score_acting': 1.309999942779541, 'raw_score_stable': 1.309999942779541, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 80, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 2260220328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 2260220328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 2260220328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 2260220328}]}, 'public_addr': '172.21.15.53:6817/2260220328', 'cluster_addr': '172.21.15.53:6819/2260220328', 'heartbeat_back_addr': '172.21.15.53:6823/2260220328', 'heartbeat_front_addr': '172.21.15.53:6821/2260220328', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 87, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 81, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:47:19.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:47:19.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:19.331 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:19.329+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:19.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:19.417 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:19.415+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:19.442 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:19.442 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:19.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:19.504 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:19.513 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:19.511+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:19.522 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:19.523 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:19.556 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:19.556 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:19.577 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:19.577 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:19.595 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:19.595 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:19.612 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:19.615 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:19.612+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:19.616 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:19.616 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:19.617 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:19.617 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:19.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:19.635 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:19.635 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:47:19.647 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-17T21:47:18.915931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1205241969,"num_objects":876,"num_object_clones":195,"num_object_copies":1752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":876,"num_whiteouts":231,"num_read":142279,"num_read_kb":875467,"num_write":10777,"num_write_kb":339660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2296,"num_bytes_recovered":319489260,"num_keys_recovered":513,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8934,"ondisk_log_size":8934,"up":82,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":756988,"kb_used_data":533744,"kb_used_omap":9,"kb_used_meta":223094,"kb_avail":565474052,"statfs":{"total":579820584960,"available":579045429248,"internally_reserved":0,"allocated":546553856,"data_stored":831266042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":228448928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10383269,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":3959,"num_read_kb":27935,"num_write":334,"num_write_kb":5342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":20355901,"num_keys_recovered":35,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016964"},"pg_stats":[{"pgid":"3.1f","version":"86'179","reported_seq":3365,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:16.115125+0000","last_change":"2024-09-17T21:46:56.830141+0000","last_active":"2024-09-17T21:47:16.115125+0000","last_peered":"2024-09-17T21:47:16.115125+0000","last_clean":"2024-09-17T21:47:16.115125+0000","last_became_active":"2024-09-17T21:46:56.828704+0000","last_became_peered":"2024-09-17T21:46:56.828704+0000","last_unstale":"2024-09-17T21:47:16.115125+0000","last_undegraded":"2024-09-17T21:47:16.115125+0000","last_fullsized":"2024-09-17T21:47:16.115125+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:43:36.475526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":41164341,"num_objects":29,"num_object_clones":3,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":9,"num_read":4719,"num_read_kb":42578,"num_write":173,"num_write_kb":8228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":10178260,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"86'257","reported_seq":6744,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:16.008636+0000","last_change":"2024-09-17T21:46:03.271053+0000","last_active":"2024-09-17T21:47:16.008636+0000","last_peered":"2024-09-17T21:47:16.008636+0000","last_clean":"2024-09-17T21:47:16.008636+0000","last_became_active":"2024-09-17T21:46:01.267064+0000","last_became_peered":"2024-09-17T21:46:01.267064+0000","last_unstale":"2024-09-17T21:47:16.008636+0000","last_undegraded":"2024-09-17T21:47:16.008636+0000","last_fullsized":"2024-09-17T21:47:16.008636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:26.273974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":94536224,"num_objects":44,"num_object_clones":13,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":7166,"num_read_kb":62609,"num_write":233,"num_write_kb":26895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"79'141","reported_seq":2568,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907387+0000","last_change":"2024-09-17T21:47:10.889674+0000","last_active":"2024-09-17T21:47:15.907387+0000","last_peered":"2024-09-17T21:47:15.907387+0000","last_clean":"2024-09-17T21:47:15.907387+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:47:15.907387+0000","last_undegraded":"2024-09-17T21:47:15.907387+0000","last_fullsized":"2024-09-17T21:47:15.907387+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:41.299331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":16869449,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":6,"num_read":2425,"num_read_kb":24378,"num_write":196,"num_write_kb":6154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"83'210","reported_seq":5081,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.909354+0000","last_change":"2024-09-17T21:47:11.890576+0000","last_active":"2024-09-17T21:47:15.909354+0000","last_peered":"2024-09-17T21:47:15.909354+0000","last_clean":"2024-09-17T21:47:15.909354+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:47:15.909354+0000","last_undegraded":"2024-09-17T21:47:15.909354+0000","last_fullsized":"2024-09-17T21:47:15.909354+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:25.254265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":65113653,"num_objects":31,"num_object_clones":10,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":4,"num_read":4695,"num_read_kb":54629,"num_write":290,"num_write_kb":33567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"86'254","reported_seq":4166,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:16.109946+0000","last_change":"2024-09-17T21:47:10.889820+0000","last_active":"2024-09-17T21:47:16.109946+0000","last_peered":"2024-09-17T21:47:16.109946+0000","last_clean":"2024-09-17T21:47:16.109946+0000","last_became_active":"2024-09-17T21:47:10.889249+0000","last_became_peered":"2024-09-17T21:47:10.889249+0000","last_unstale":"2024-09-17T21:47:16.109946+0000","last_undegraded":"2024-09-17T21:47:16.109946+0000","last_fullsized":"2024-09-17T21:47:16.109946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:58:24.166422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":49235968,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":5,"num_read":5241,"num_read_kb":19433,"num_write":195,"num_write_kb":5020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":9789440,"num_keys_recovered":54,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"84'285","reported_seq":5356,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T21:47:14.900340+0000","last_change":"2024-09-17T21:47:10.889765+0000","last_active":"2024-09-17T21:47:14.900340+0000","last_peered":"2024-09-17T21:47:14.900340+0000","last_clean":"2024-09-17T21:47:14.900340+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:47:14.900340+0000","last_undegraded":"2024-09-17T21:47:14.900340+0000","last_fullsized":"2024-09-17T21:47:14.900340+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:12.242941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":60685994,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":5,"num_read":6611,"num_read_kb":43263,"num_write":299,"num_write_kb":18431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"76'165","reported_seq":2695,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.892736+0000","last_change":"2024-09-17T21:47:11.890115+0000","last_active":"2024-09-17T21:47:13.892736+0000","last_peered":"2024-09-17T21:47:13.892736+0000","last_clean":"2024-09-17T21:47:13.892736+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:47:13.892736+0000","last_undegraded":"2024-09-17T21:47:13.892736+0000","last_fullsized":"2024-09-17T21:47:13.892736+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:02:05.437861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":26772016,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":2218,"num_read_kb":16591,"num_write":169,"num_write_kb":7911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"86'412","reported_seq":3563,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901518+0000","last_change":"2024-09-17T21:46:56.829581+0000","last_active":"2024-09-17T21:47:15.901518+0000","last_peered":"2024-09-17T21:47:15.901518+0000","last_clean":"2024-09-17T21:47:15.901518+0000","last_became_active":"2024-09-17T21:46:56.828057+0000","last_became_peered":"2024-09-17T21:46:56.828057+0000","last_unstale":"2024-09-17T21:47:15.901518+0000","last_undegraded":"2024-09-17T21:47:15.901518+0000","last_fullsized":"2024-09-17T21:47:15.901518+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:18.653395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":41489063,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":8,"num_read":4177,"num_read_kb":14377,"num_write":568,"num_write_kb":4135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":4195404,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"86'139","reported_seq":2100,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.904452+0000","last_change":"2024-09-17T21:46:56.830587+0000","last_active":"2024-09-17T21:47:15.904452+0000","last_peered":"2024-09-17T21:47:15.904452+0000","last_clean":"2024-09-17T21:47:15.904452+0000","last_became_active":"2024-09-17T21:46:56.829387+0000","last_became_peered":"2024-09-17T21:46:56.829387+0000","last_unstale":"2024-09-17T21:47:15.904452+0000","last_undegraded":"2024-09-17T21:47:15.904452+0000","last_fullsized":"2024-09-17T21:47:15.904452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:03:56.704655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":11492864,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":2015,"num_read_kb":18302,"num_write":162,"num_write_kb":4962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5990912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"86'254","reported_seq":3818,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907385+0000","last_change":"2024-09-17T21:46:03.282056+0000","last_active":"2024-09-17T21:47:15.907385+0000","last_peered":"2024-09-17T21:47:15.907385+0000","last_clean":"2024-09-17T21:47:15.907385+0000","last_became_active":"2024-09-17T21:46:01.267932+0000","last_became_peered":"2024-09-17T21:46:01.267932+0000","last_unstale":"2024-09-17T21:47:15.907385+0000","last_undegraded":"2024-09-17T21:47:15.907385+0000","last_fullsized":"2024-09-17T21:47:15.907385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:55:18.617580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":19639331,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":9,"num_read":4774,"num_read_kb":33303,"num_write":227,"num_write_kb":12762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"86'434","reported_seq":6395,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901496+0000","last_change":"2024-09-17T21:46:56.832432+0000","last_active":"2024-09-17T21:47:15.901496+0000","last_peered":"2024-09-17T21:47:15.901496+0000","last_clean":"2024-09-17T21:47:15.901496+0000","last_became_active":"2024-09-17T21:46:56.831415+0000","last_became_peered":"2024-09-17T21:46:56.831415+0000","last_unstale":"2024-09-17T21:47:15.901496+0000","last_undegraded":"2024-09-17T21:47:15.901496+0000","last_fullsized":"2024-09-17T21:47:15.901496+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:31.302065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":20616739,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":8339,"num_read_kb":16663,"num_write":400,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":5601792,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"83'169","reported_seq":4918,"reported_epoch":85,"state":"remapped+peering","last_fresh":"2024-09-17T21:47:14.900770+0000","last_change":"2024-09-17T21:47:14.900770+0000","last_active":"2024-09-17T21:47:13.894980+0000","last_peered":"2024-09-17T21:47:13.578505+0000","last_clean":"2024-09-17T21:47:13.578505+0000","last_became_active":"2024-09-17T21:47:11.891507+0000","last_became_peered":"2024-09-17T21:47:11.891507+0000","last_unstale":"2024-09-17T21:47:14.900770+0000","last_undegraded":"2024-09-17T21:47:14.900770+0000","last_fullsized":"2024-09-17T21:47:14.900770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":83551340,"num_objects":42,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":9,"num_read":4317,"num_read_kb":59944,"num_write":177,"num_write_kb":15414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":113,"num_bytes_recovered":26126972,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":162,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901596+0000","last_change":"2024-09-17T21:47:07.868723+0000","last_active":"2024-09-17T21:47:15.901596+0000","last_peered":"2024-09-17T21:47:15.901596+0000","last_clean":"2024-09-17T21:47:15.901596+0000","last_became_active":"2024-09-17T21:47:07.868434+0000","last_became_peered":"2024-09-17T21:47:07.868434+0000","last_unstale":"2024-09-17T21:47:15.901596+0000","last_undegraded":"2024-09-17T21:47:15.901596+0000","last_fullsized":"2024-09-17T21:47:15.901596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:25.706905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"83'134","reported_seq":2908,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.904659+0000","last_change":"2024-09-17T21:46:56.829051+0000","last_active":"2024-09-17T21:47:15.904659+0000","last_peered":"2024-09-17T21:47:15.904659+0000","last_clean":"2024-09-17T21:47:15.904659+0000","last_became_active":"2024-09-17T21:46:56.828260+0000","last_became_peered":"2024-09-17T21:46:56.828260+0000","last_unstale":"2024-09-17T21:47:15.904659+0000","last_undegraded":"2024-09-17T21:47:15.904659+0000","last_fullsized":"2024-09-17T21:47:15.904659+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:45:14.758333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":28854311,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":8,"num_read":2369,"num_read_kb":17585,"num_write":183,"num_write_kb":7002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":12296387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":149,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.893106+0000","last_change":"2024-09-17T21:45:54.240924+0000","last_active":"2024-09-17T21:47:13.893106+0000","last_peered":"2024-09-17T21:47:13.893106+0000","last_clean":"2024-09-17T21:47:13.893106+0000","last_became_active":"2024-09-17T21:45:52.236111+0000","last_became_peered":"2024-09-17T21:45:52.236111+0000","last_unstale":"2024-09-17T21:47:13.893106+0000","last_undegraded":"2024-09-17T21:47:13.893106+0000","last_fullsized":"2024-09-17T21:47:13.893106+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:14:53.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"86'262","reported_seq":2383,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:16.083120+0000","last_change":"2024-09-17T21:46:03.270712+0000","last_active":"2024-09-17T21:47:16.083120+0000","last_peered":"2024-09-17T21:47:16.083120+0000","last_clean":"2024-09-17T21:47:16.083120+0000","last_became_active":"2024-09-17T21:46:01.265693+0000","last_became_peered":"2024-09-17T21:46:01.265693+0000","last_unstale":"2024-09-17T21:47:16.083120+0000","last_undegraded":"2024-09-17T21:47:16.083120+0000","last_fullsized":"2024-09-17T21:47:16.083120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:51:11.323203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":33386526,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":10,"num_read":2561,"num_read_kb":18881,"num_write":345,"num_write_kb":6447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":177,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901634+0000","last_change":"2024-09-17T21:46:53.656743+0000","last_active":"2024-09-17T21:47:15.901634+0000","last_peered":"2024-09-17T21:47:15.901634+0000","last_clean":"2024-09-17T21:47:15.901634+0000","last_became_active":"2024-09-17T21:46:53.656349+0000","last_became_peered":"2024-09-17T21:46:53.656349+0000","last_unstale":"2024-09-17T21:47:15.901634+0000","last_undegraded":"2024-09-17T21:47:15.901634+0000","last_fullsized":"2024-09-17T21:47:15.901634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:42.054415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"79'213","reported_seq":4975,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.892852+0000","last_change":"2024-09-17T21:46:03.279109+0000","last_active":"2024-09-17T21:47:13.892852+0000","last_peered":"2024-09-17T21:47:13.892852+0000","last_clean":"2024-09-17T21:47:13.892852+0000","last_became_active":"2024-09-17T21:46:01.268063+0000","last_became_peered":"2024-09-17T21:46:01.268063+0000","last_unstale":"2024-09-17T21:47:13.892852+0000","last_undegraded":"2024-09-17T21:47:13.892852+0000","last_fullsized":"2024-09-17T21:47:13.892852+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:33:02.984785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":61194240,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":9,"num_read":5488,"num_read_kb":36915,"num_write":187,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.892954+0000","last_change":"2024-09-17T21:46:53.661449+0000","last_active":"2024-09-17T21:47:13.892954+0000","last_peered":"2024-09-17T21:47:13.892954+0000","last_clean":"2024-09-17T21:47:13.892954+0000","last_became_active":"2024-09-17T21:46:53.661123+0000","last_became_peered":"2024-09-17T21:46:53.661123+0000","last_unstale":"2024-09-17T21:47:13.892954+0000","last_undegraded":"2024-09-17T21:47:13.892954+0000","last_fullsized":"2024-09-17T21:47:13.892954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:05.789195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"83'216","reported_seq":2790,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907512+0000","last_change":"2024-09-17T21:46:03.282656+0000","last_active":"2024-09-17T21:47:15.907512+0000","last_peered":"2024-09-17T21:47:15.907512+0000","last_clean":"2024-09-17T21:47:15.907512+0000","last_became_active":"2024-09-17T21:46:01.271756+0000","last_became_peered":"2024-09-17T21:46:01.271756+0000","last_unstale":"2024-09-17T21:47:15.907512+0000","last_undegraded":"2024-09-17T21:47:15.907512+0000","last_fullsized":"2024-09-17T21:47:15.907512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:44.269120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":26604064,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":3602,"num_read_kb":14042,"num_write":174,"num_write_kb":5068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"84'552","reported_seq":2487,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T21:47:14.900475+0000","last_change":"2024-09-17T21:47:11.890882+0000","last_active":"2024-09-17T21:47:14.900475+0000","last_peered":"2024-09-17T21:47:14.900475+0000","last_clean":"2024-09-17T21:47:14.900475+0000","last_became_active":"2024-09-17T21:47:11.889462+0000","last_became_peered":"2024-09-17T21:47:11.889462+0000","last_unstale":"2024-09-17T21:47:14.900475+0000","last_undegraded":"2024-09-17T21:47:14.900475+0000","last_fullsized":"2024-09-17T21:47:14.900475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":552,"log_dups_size":0,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:54:42.498079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":20365236,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":4132,"num_read_kb":9200,"num_write":815,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":155,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907235+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:47:15.907235+0000","last_peered":"2024-09-17T21:47:15.907235+0000","last_clean":"2024-09-17T21:47:15.907235+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:47:15.907235+0000","last_undegraded":"2024-09-17T21:47:15.907235+0000","last_fullsized":"2024-09-17T21:47:15.907235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"79'153","reported_seq":2553,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.893054+0000","last_change":"2024-09-17T21:46:03.278873+0000","last_active":"2024-09-17T21:47:13.893054+0000","last_peered":"2024-09-17T21:47:13.893054+0000","last_clean":"2024-09-17T21:47:13.893054+0000","last_became_active":"2024-09-17T21:46:01.266414+0000","last_became_peered":"2024-09-17T21:46:01.266414+0000","last_unstale":"2024-09-17T21:47:13.893054+0000","last_undegraded":"2024-09-17T21:47:13.893054+0000","last_fullsized":"2024-09-17T21:47:13.893054+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:48:15.022964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":17318011,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":4,"num_read":3111,"num_read_kb":11004,"num_write":150,"num_write_kb":2585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":153,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.902230+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:47:15.902230+0000","last_peered":"2024-09-17T21:47:15.902230+0000","last_clean":"2024-09-17T21:47:15.902230+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:47:15.902230+0000","last_undegraded":"2024-09-17T21:47:15.902230+0000","last_fullsized":"2024-09-17T21:47:15.902230+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"83'1192","reported_seq":4529,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.904241+0000","last_change":"2024-09-17T21:47:11.891646+0000","last_active":"2024-09-17T21:47:15.904241+0000","last_peered":"2024-09-17T21:47:15.904241+0000","last_clean":"2024-09-17T21:47:15.904241+0000","last_became_active":"2024-09-17T21:47:11.890047+0000","last_became_peered":"2024-09-17T21:47:11.890047+0000","last_unstale":"2024-09-17T21:47:15.904241+0000","last_undegraded":"2024-09-17T21:47:15.904241+0000","last_fullsized":"2024-09-17T21:47:15.904241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1192,"log_dups_size":0,"ondisk_log_size":1192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:35:54.012197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":17131311,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":10,"num_read":9361,"num_read_kb":26651,"num_write":1961,"num_write_kb":8269,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7135373,"num_keys_recovered":83,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":152,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901305+0000","last_change":"2024-09-17T21:45:54.626274+0000","last_active":"2024-09-17T21:47:15.901305+0000","last_peered":"2024-09-17T21:47:15.901305+0000","last_clean":"2024-09-17T21:47:15.901305+0000","last_became_active":"2024-09-17T21:45:52.236559+0000","last_became_peered":"2024-09-17T21:45:52.236559+0000","last_unstale":"2024-09-17T21:47:15.901305+0000","last_undegraded":"2024-09-17T21:47:15.901305+0000","last_fullsized":"2024-09-17T21:47:15.901305+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:27.009628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000474976,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"84'222","reported_seq":2891,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.892988+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:47:13.892988+0000","last_peered":"2024-09-17T21:47:13.892988+0000","last_clean":"2024-09-17T21:47:13.892988+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:47:13.892988+0000","last_undegraded":"2024-09-17T21:47:13.892988+0000","last_fullsized":"2024-09-17T21:47:13.892988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:20:10.636129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":39269438,"num_objects":27,"num_object_clones":5,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":7,"num_read":3115,"num_read_kb":25500,"num_write":298,"num_write_kb":17674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":154,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907366+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:47:15.907366+0000","last_peered":"2024-09-17T21:47:15.907366+0000","last_clean":"2024-09-17T21:47:15.907366+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:47:15.907366+0000","last_undegraded":"2024-09-17T21:47:15.907366+0000","last_fullsized":"2024-09-17T21:47:15.907366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":355,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901312+0000","last_change":"2024-09-17T21:46:56.831645+0000","last_active":"2024-09-17T21:47:15.901312+0000","last_peered":"2024-09-17T21:47:15.901312+0000","last_clean":"2024-09-17T21:47:15.901312+0000","last_became_active":"2024-09-17T21:46:56.831300+0000","last_became_peered":"2024-09-17T21:46:56.831300+0000","last_unstale":"2024-09-17T21:47:15.901312+0000","last_undegraded":"2024-09-17T21:47:15.901312+0000","last_fullsized":"2024-09-17T21:47:15.901312+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:31:03.273489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"76'149","reported_seq":3181,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901531+0000","last_change":"2024-09-17T21:46:03.270987+0000","last_active":"2024-09-17T21:47:15.901531+0000","last_peered":"2024-09-17T21:47:15.901531+0000","last_clean":"2024-09-17T21:47:15.901531+0000","last_became_active":"2024-09-17T21:46:01.270323+0000","last_became_peered":"2024-09-17T21:46:01.270323+0000","last_unstale":"2024-09-17T21:47:15.901531+0000","last_undegraded":"2024-09-17T21:47:15.901531+0000","last_fullsized":"2024-09-17T21:47:15.901531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:11:32.099195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":25878560,"num_objects":26,"num_object_clones":4,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":3459,"num_read_kb":14466,"num_write":144,"num_write_kb":1984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"84'233","reported_seq":4197,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.087623+0000","last_change":"2024-09-17T21:47:11.890286+0000","last_active":"2024-09-17T21:47:15.087623+0000","last_peered":"2024-09-17T21:47:15.087623+0000","last_clean":"2024-09-17T21:47:15.087623+0000","last_became_active":"2024-09-17T21:47:11.889243+0000","last_became_peered":"2024-09-17T21:47:11.889243+0000","last_unstale":"2024-09-17T21:47:15.087623+0000","last_undegraded":"2024-09-17T21:47:15.087623+0000","last_fullsized":"2024-09-17T21:47:15.087623+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":233,"log_dups_size":0,"ondisk_log_size":233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:09.612006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":43611738,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":4506,"num_read_kb":37804,"num_write":254,"num_write_kb":21335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":34502270,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"83'162","reported_seq":2193,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901421+0000","last_change":"2024-09-17T21:47:10.890100+0000","last_active":"2024-09-17T21:47:15.901421+0000","last_peered":"2024-09-17T21:47:15.901421+0000","last_clean":"2024-09-17T21:47:15.901421+0000","last_became_active":"2024-09-17T21:47:10.889505+0000","last_became_peered":"2024-09-17T21:47:10.889505+0000","last_unstale":"2024-09-17T21:47:15.901421+0000","last_undegraded":"2024-09-17T21:47:15.901421+0000","last_fullsized":"2024-09-17T21:47:15.901421+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:32.515733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":4583502,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":3,"num_read":1843,"num_read_kb":7139,"num_write":179,"num_write_kb":1138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":721052,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"83'235","reported_seq":5498,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-17T21:47:13.892796+0000","last_change":"2024-09-17T21:46:56.829501+0000","last_active":"2024-09-17T21:47:13.892796+0000","last_peered":"2024-09-17T21:47:13.892796+0000","last_clean":"2024-09-17T21:47:13.892796+0000","last_became_active":"2024-09-17T21:46:56.828435+0000","last_became_peered":"2024-09-17T21:46:56.828435+0000","last_unstale":"2024-09-17T21:47:13.892796+0000","last_undegraded":"2024-09-17T21:47:13.892796+0000","last_fullsized":"2024-09-17T21:47:13.892796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:31:52.836306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":26746956,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":8,"num_read":6375,"num_read_kb":33559,"num_write":226,"num_write_kb":7872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":13386408,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"83'247","reported_seq":2485,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.994868+0000","last_change":"2024-09-17T21:46:56.832225+0000","last_active":"2024-09-17T21:47:15.994868+0000","last_peered":"2024-09-17T21:47:15.994868+0000","last_clean":"2024-09-17T21:47:15.994868+0000","last_became_active":"2024-09-17T21:46:56.831587+0000","last_became_peered":"2024-09-17T21:46:56.831587+0000","last_unstale":"2024-09-17T21:47:15.994868+0000","last_undegraded":"2024-09-17T21:47:15.994868+0000","last_fullsized":"2024-09-17T21:47:15.994868+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:13.160280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":21632618,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":2061,"num_read_kb":10728,"num_write":284,"num_write_kb":5197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"86'553","reported_seq":8526,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901598+0000","last_change":"2024-09-17T21:46:03.272242+0000","last_active":"2024-09-17T21:47:15.901598+0000","last_peered":"2024-09-17T21:47:15.901598+0000","last_clean":"2024-09-17T21:47:15.901598+0000","last_became_active":"2024-09-17T21:46:01.268272+0000","last_became_peered":"2024-09-17T21:46:01.268272+0000","last_unstale":"2024-09-17T21:47:15.901598+0000","last_undegraded":"2024-09-17T21:47:15.901598+0000","last_fullsized":"2024-09-17T21:47:15.901598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":553,"log_dups_size":0,"ondisk_log_size":553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:06:34.376916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":107205199,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":9968,"num_read_kb":65593,"num_write":630,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"86'110","reported_seq":2153,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:16.084461+0000","last_change":"2024-09-17T21:47:07.910393+0000","last_active":"2024-09-17T21:47:16.084461+0000","last_peered":"2024-09-17T21:47:16.084461+0000","last_clean":"2024-09-17T21:47:16.084461+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:47:16.084461+0000","last_undegraded":"2024-09-17T21:47:16.084461+0000","last_fullsized":"2024-09-17T21:47:16.084461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:18:51.083305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":12293684,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1770,"num_read_kb":8068,"num_write":125,"num_write_kb":2523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"86'472","reported_seq":7111,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.907490+0000","last_change":"2024-09-17T21:46:03.282498+0000","last_active":"2024-09-17T21:47:15.907490+0000","last_peered":"2024-09-17T21:47:15.907490+0000","last_clean":"2024-09-17T21:47:15.907490+0000","last_became_active":"2024-09-17T21:46:01.268915+0000","last_became_peered":"2024-09-17T21:46:01.268915+0000","last_unstale":"2024-09-17T21:47:15.907490+0000","last_undegraded":"2024-09-17T21:47:15.907490+0000","last_fullsized":"2024-09-17T21:47:15.907490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:55.851059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":31867537,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":9526,"num_read_kb":49918,"num_write":653,"num_write_kb":26807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"86'313","reported_seq":5567,"reported_epoch":86,"state":"active+undersized+remapped","last_fresh":"2024-09-17T21:47:16.089921+0000","last_change":"2024-09-17T21:47:15.983594+0000","last_active":"2024-09-17T21:47:16.089921+0000","last_peered":"2024-09-17T21:47:16.089921+0000","last_clean":"2024-09-17T21:47:13.849653+0000","last_became_active":"2024-09-17T21:47:15.911964+0000","last_became_peered":"2024-09-17T21:47:15.911964+0000","last_unstale":"2024-09-17T21:47:16.089921+0000","last_undegraded":"2024-09-17T21:47:16.089921+0000","last_fullsized":"2024-09-17T21:47:15.904210+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":313,"log_dups_size":0,"ondisk_log_size":313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":33622831,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":6393,"num_read_kb":17672,"num_write":338,"num_write_kb":3196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":4987381,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0","objects":31}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"76'189","reported_seq":3343,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901322+0000","last_change":"2024-09-17T21:47:11.892531+0000","last_active":"2024-09-17T21:47:15.901322+0000","last_peered":"2024-09-17T21:47:15.901322+0000","last_clean":"2024-09-17T21:47:15.901322+0000","last_became_active":"2024-09-17T21:47:11.891574+0000","last_became_peered":"2024-09-17T21:47:11.891574+0000","last_unstale":"2024-09-17T21:47:15.901322+0000","last_undegraded":"2024-09-17T21:47:15.901322+0000","last_fullsized":"2024-09-17T21:47:15.901322+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:06:43.783748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":66662960,"num_objects":37,"num_object_clones":13,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":13,"num_read":2696,"num_read_kb":39220,"num_write":200,"num_write_kb":24336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":63162512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"86'146","reported_seq":802,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.956997+0000","last_change":"2024-09-17T21:46:56.829405+0000","last_active":"2024-09-17T21:47:15.956997+0000","last_peered":"2024-09-17T21:47:15.956997+0000","last_clean":"2024-09-17T21:47:15.956997+0000","last_became_active":"2024-09-17T21:46:56.828762+0000","last_became_peered":"2024-09-17T21:46:56.828762+0000","last_unstale":"2024-09-17T21:47:15.956997+0000","last_undegraded":"2024-09-17T21:47:15.956997+0000","last_fullsized":"2024-09-17T21:47:15.956997+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:13.075057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":20325964,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":6,"num_read":630,"num_read_kb":7625,"num_write":176,"num_write_kb":4014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3249740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"76'162","reported_seq":2567,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-17T21:47:15.901357+0000","last_change":"2024-09-17T21:46:03.272109+0000","last_active":"2024-09-17T21:47:15.901357+0000","last_peered":"2024-09-17T21:47:15.901357+0000","last_clean":"2024-09-17T21:47:15.901357+0000","last_became_active":"2024-09-17T21:46:01.267697+0000","last_became_peered":"2024-09-17T21:46:01.267697+0000","last_unstale":"2024-09-17T21:47:15.901357+0000","last_undegraded":"2024-09-17T21:47:15.901357+0000","last_fullsized":"2024-09-17T21:47:15.901357+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:22:51.392681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":34929914,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":2500,"num_read_kb":17727,"num_write":205,"num_write_kb":4764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1204651582,"num_objects":872,"num_object_clones":195,"num_object_copies":1744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":872,"num_whiteouts":231,"num_read":142163,"num_read_kb":875367,"num_write":10616,"num_write_kb":337242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2288,"num_bytes_recovered":317127788,"num_keys_recovered":513,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":551821312,"data_stored":846088424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8814,"ondisk_log_size":8814,"up":64,"acting":62,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":2361472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607574,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121768,"kb_used_data":94732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94250072,"statfs":{"total":96636764160,"available":96512073728,"internally_reserved":0,"allocated":97005568,"data_stored":163859713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607573,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161128,"kb_used_data":129940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94210712,"statfs":{"total":96636764160,"available":96471769088,"internally_reserved":0,"allocated":133058560,"data_stored":201912418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607573,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":131136,"kb_used_data":81896,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94240704,"statfs":{"total":96636764160,"available":96502480896,"internally_reserved":0,"allocated":83861504,"data_stored":130531552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607573,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77788,"kb_used_data":46592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94294052,"statfs":{"total":96636764160,"available":96557109248,"internally_reserved":0,"allocated":47710208,"data_stored":59667697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607573,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":170624,"kb_used_data":117228,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94201216,"statfs":{"total":96636764160,"available":96462045184,"internally_reserved":0,"allocated":120041472,"data_stored":183299729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640277,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":94544,"kb_used_data":63356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94277296,"statfs":{"total":96636764160,"available":96539951104,"internally_reserved":0,"allocated":64876544,"data_stored":91994933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45211648,"data_stored":57777722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":63893504,"data_stored":91596868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82759680,"data_stored":130030665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131280896,"data_stored":200747947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":119062528,"data_stored":182901683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":109613056,"data_stored":183033539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:47:19.650 INFO:tasks.thrashosds.thrasher:Setting 3.1b to [5, 2, 4, 1] 2024-09-17T21:47:19.650 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1b', '5', '2', '4', '1'] 2024-09-17T21:47:19.650 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.1b 5 2 4 1 2024-09-17T21:47:19.731 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:19.729+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.058085445000003, 2024-09-17T21:47:19.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:19.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:19.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.025810470000003, 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.025571702000001, 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:19.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:19.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 26.880862087000001, 2024-09-17T21:47:19.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:19.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:627 3.6 3:6244dd87:::rbd_data.1112bc1b2568.0000000000000019:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.790342354, 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030691610000000001, 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 627 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950565+0000", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950567+0000", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950574+0000", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950580+0000", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950631+0000", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1863999999999997e-05 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950752+0000", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120596 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981256+0000", 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030503834000000001 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:745 3.6 3:60bbb74c:::rbd_data.1112bc1b2568.00000000000000de:head [delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.782433328, 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023010439000000001, 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 745 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958474+0000", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958476+0000", 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2299999999999998e-06 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958482+0000", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958488+0000", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961234+0000", 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027465929999999999 2024-09-17T21:47:19.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961306+0000", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1521999999999997e-05 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961496+0000", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190053 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.962386+0000", 2024-09-17T21:47:19.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089013799999999995 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981403+0000", 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019016943000000001 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981448+0000", 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5212e-05 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981484+0000", 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5608999999999997e-05 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1235 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753876804000001, 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013460730000000001, 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1235 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987032+0000", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9209999999999999e-06 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987043+0000", 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1076e-05 2024-09-17T21:47:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987049+0000", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000331+0000", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013281866999999999 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000405+0000", 2024-09-17T21:47:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4004000000000005e-05 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000491+0000", 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6062000000000004e-05 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1236 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753850874999999, 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887803000000001, 2024-09-17T21:47:19.779 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1236 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987058+0000", 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987065+0000", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1219999999999997e-06 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987070+0000", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9529999999999998e-06 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000520+0000", 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013449796999999999 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000592+0000", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919e-05 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000944+0000", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035145700000000001 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1237 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:19.783 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753829767999999, 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049556, 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1237 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:19.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987080+0000", 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:19.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987086+0000", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6309999999999997e-06 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987091+0000", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000974+0000", 2024-09-17T21:47:19.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013882864999999999 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001045+0000", 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0661000000000007e-05 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001126+0000", 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1521999999999996e-05 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1264 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753658848000001, 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014065061, 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1264 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987249+0000", 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:47:19.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987252+0000", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987263+0000", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1289e-05 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987270+0000", 2024-09-17T21:47:19.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1330000000000001e-06 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001154+0000", 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01388413 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001225+0000", 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0975000000000004e-05 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001313+0000", 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7404999999999994e-05 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1265 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753625538, 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01421443, 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1265 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987282+0000", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987284+0000", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.328e-06 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987291+0000", 2024-09-17T21:47:19.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987305+0000", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3344e-05 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001341+0000", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014035903000000001 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001411+0000", 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0502999999999999e-05 2024-09-17T21:47:19.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001496+0000", 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4383999999999997e-05 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1266 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753593598, 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014670829, 2024-09-17T21:47:19.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1266 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987314+0000", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987316+0000", 2024-09-17T21:47:19.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4059999999999999e-06 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987323+0000", 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987334+0000", 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.093e-05 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001524+0000", 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014189715 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001592+0000", 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8132000000000004e-05 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001984+0000", 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039185199999999999 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1267 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753564627999999, 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014824201, 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1267 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987343+0000", 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:19.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987345+0000", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987351+0000", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987356+0000", 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:47:19.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002014+0000", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014657411 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002084+0000", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0596e-05 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002166+0000", 2024-09-17T21:47:19.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2083999999999995e-05 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1268 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753542979000001, 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014992027999999999, 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1268 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987369+0000", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7129999999999996e-06 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987376+0000", 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987381+0000", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002195+0000", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014814226 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002268+0000", 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3279000000000001e-05 2024-09-17T21:47:19.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002356+0000", 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7305999999999994e-05 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1269 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.753515815, 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015291325999999999, 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1269 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987394+0000", 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3989999999999998e-06 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987401+0000", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8210000000000004e-06 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987406+0000", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0030000000000002e-06 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002522+0000", 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015116368 2024-09-17T21:47:19.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002595+0000", 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3335000000000002e-05 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002682+0000", 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6927000000000003e-05 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:771 3.1 3:8006aab1:::rbd_data.1112f0bef54a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 8280576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.136762762, 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013777717, 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 771 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604145+0000", 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608094+0000", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039492950000000002 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608114+0000", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0254e-05 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608122+0000", 2024-09-17T21:47:19.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608205+0000", 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3511999999999996e-05 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608304+0000", 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8524999999999996e-05 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608621+0000", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031733999999999999 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617430+0000", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088086459999999998 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617787+0000", 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035710100000000002 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617854+0000", 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7699000000000004e-05 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617922+0000", 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7169999999999996e-05 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:19.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8580655719999992, 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8511638010000002, 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:19.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8510477099999996, 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:19.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:19.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:19.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2165 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch ping cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.5448126309999992, 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69677434100000002, 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:19.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2165 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:19.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196096+0000", 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196108+0000", 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196130+0000", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1695000000000002e-05 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196140+0000", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196216+0000", 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899e-05 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196227+0000", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1008e-05 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892718+0000", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69649070000000002 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892759+0000", 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1876999999999999e-05 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:19.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892868+0000", 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010892599999999999 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:19.840 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:19.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:19.842 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:19.839+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:19.942 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:19.939+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:19.995 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:19.995 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:19.995 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:20.043 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:20.040+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:20.050 INFO:teuthology.orchestra.run.smithi165.stdout:flattening image image_client.0-clone26 2024-09-17T21:47:20.115 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:20.115 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:20.116 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:20.168 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:20.166+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:20.234 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327909167999998, 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.235 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:20.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327898867000002, 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.239 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:20.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327882793000001, 2024-09-17T21:47:20.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:20.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.32787158, 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:20.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:20.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:20.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327854142, 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327843272000003, 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:20.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:20.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.327648394999997, 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:20.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.325943725000002, 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:20.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:20.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.325933087000003, 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:20.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.325922994000003, 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.325912875, 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:20.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:20.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:20.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.323782645000001, 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:20.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:20.285 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.323772914000003, 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.322989554999999, 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.322973984999997, 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:20.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:20.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.322881082999999, 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:20.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.322386463999997, 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.306 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.322242418999998, 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:20.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:20.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:20.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.321556981999997, 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:20.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:20.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.321194591999998, 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.319 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:20.321 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:20.318+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:20.392 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-17T21:47:20.421 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:20.419+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:20.471 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:20.471 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:20.471 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:20.471 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.479 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:20.522 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:20.520+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:20.622 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:20.620+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:20.633 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:20.633 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:20.633 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:20.723 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:20.721+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.945866451000001, 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:47:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.936215486999998, 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.852269618000001, 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:47:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:47:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.850807482999997, 2024-09-17T21:47:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:47:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:47:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:47:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.848196365, 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.784081907999997, 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.783353685999998, 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:47:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:47:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:47:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.449776550000003, 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:47:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.883018047999997, 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.875495905999998, 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:47:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:47:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1128 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.765206834000001, 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099832400000000009, 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1128 2024-09-17T21:47:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990586+0000", 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.125e-06 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990593+0000", 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6919999999999996e-06 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990598+0000", 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9830000000000004e-06 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000414+0000", 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098157309999999994 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000481+0000", 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7303000000000003e-05 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000567+0000", 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5923999999999999e-05 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1129 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.765186283, 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010143592999999999, 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1129 2024-09-17T21:47:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990605+0000", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990607+0000", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990613+0000", 2024-09-17T21:47:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990618+0000", 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000596+0000", 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099777809999999998 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000662+0000", 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5891999999999999e-05 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000748+0000", 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6372999999999994e-05 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1224 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.758336849999999, 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010061038, 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1224 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997457+0000", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3499999999999999e-06 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998930+0000", 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014735079999999999 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998947+0000", 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6977999999999999e-05 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007338+0000", 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083907679999999998 2024-09-17T21:47:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007417+0000", 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9008e-05 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007515+0000", 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7958e-05 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1255 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.756360938, 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010112939, 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1255 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999432+0000", 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249999999999999e-06 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999439+0000", 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999445+0000", 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2730000000000002e-06 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009368+0000", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099234240000000001 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009446+0000", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8209999999999998e-05 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009543+0000", 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6353000000000004e-05 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1256 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.756339473000001, 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010198560000000001, 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1256 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999452+0000", 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999453+0000", 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.945e-06 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999460+0000", 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7789999999999998e-06 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999469+0000", 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6309999999999992e-06 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009556+0000", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010086927000000001 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009597+0000", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1038999999999998e-05 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009650+0000", 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2831999999999998e-05 2024-09-17T21:47:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1288 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.753323154, 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010050948000000001, 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1288 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002468+0000", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002470+0000", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.384e-06 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002477+0000", 2024-09-17T21:47:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002482+0000", 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1089999999999997e-06 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012369+0000", 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098866030000000008 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012434+0000", 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5512999999999994e-05 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012518+0000", 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4089000000000001e-05 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1307 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.745305902, 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099939750000000004, 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1307 2024-09-17T21:47:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010488+0000", 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0240000000000002e-06 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010495+0000", 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010508+0000", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3410999999999999e-05 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017483+0000", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069745320000000003 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017570+0000", 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7057000000000004e-05 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017970+0000", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040009600000000001 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020303+0000", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023332729999999999 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020439+0000", 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135783 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020462+0000", 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2469000000000001e-05 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020479+0000", 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7144999999999999e-05 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1309 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.745273505, 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010826840000000001, 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1309 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010518+0000", 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010521+0000", 2024-09-17T21:47:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.207e-06 2024-09-17T21:47:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010528+0000", 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010533+0000", 2024-09-17T21:47:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1930000000000001e-06 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018747+0000", 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082139880000000002 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018835+0000", 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7560999999999997e-05 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.019193+0000", 2024-09-17T21:47:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035790399999999998 2024-09-17T21:47:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020506+0000", 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013139099999999999 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021255+0000", 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074901999999999998 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021305+0000", 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230999999999997e-05 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021344+0000", 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9252000000000003e-05 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1311 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.745250011, 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012406011, 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1311 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010544+0000", 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.977e-06 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010551+0000", 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7240000000000003e-06 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010556+0000", 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0989999999999998e-06 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020524+0000", 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.009968457 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020568+0000", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4132999999999999e-05 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020784+0000", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021525299999999999 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022838+0000", 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020542030000000001 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022848+0000", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9879999999999999e-06 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022894+0000", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5803000000000002e-05 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022947+0000", 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2896999999999999e-05 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1312 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.742867457999999, 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011273178999999999, 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:20.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1312 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012924+0000", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012937+0000", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2662e-05 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012950+0000", 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012958+0000", 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9140000000000008e-06 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021388+0000", 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084298759999999993 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:20.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021609+0000", 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221256 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021953+0000", 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034352200000000002 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022955+0000", 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010025909999999999 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024087+0000", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011314109999999999 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024148+0000", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1001000000000001e-05 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024196+0000", 2024-09-17T21:47:20.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8474999999999999e-05 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:20.873 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:20.875 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:20.876 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:20.872+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:20.924 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.1b pg_upmap_items mapping to [5->2,4->1] 2024-09-17T21:47:20.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:47:20.979 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:20.977+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:20.984 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:20.984 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:20.984 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:20.985 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:20.994 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:21.000 INFO:teuthology.orchestra.run.smithi165.stdout:3409 write 0x39afc00 thru 0x39bcf0b (0xd30c bytes) 2024-09-17T21:47:21.010 INFO:teuthology.orchestra.run.smithi165.stdout:3410 trunc from 0x39bd000 to 0xb340800 2024-09-17T21:47:21.024 INFO:teuthology.orchestra.run.smithi165.stdout:3411 punch from 0x4fe7a00 to 0x4fe7c00, (0x200 bytes) 2024-09-17T21:47:21.027 INFO:teuthology.orchestra.run.smithi165.stdout:3413 write 0x8511400 thru 0x851b2c0 (0x9ec1 bytes) 2024-09-17T21:47:21.027 INFO:teuthology.orchestra.run.smithi165.stdout:3414 write 0x851f000 thru 0x852dc6c (0xec6d bytes) 2024-09-17T21:47:21.045 INFO:teuthology.orchestra.run.smithi165.stdout:3415 write 0xeb38000 thru 0xeb3e37b (0x637c bytes) 2024-09-17T21:47:21.047 INFO:teuthology.orchestra.run.smithi165.stdout:3416 punch from 0xac77c00 to 0xac7b000, (0x3400 bytes) 2024-09-17T21:47:21.048 INFO:teuthology.orchestra.run.smithi165.stdout:3417 read 0x1f77600 thru 0x1f83d24 (0xc725 bytes) 2024-09-17T21:47:21.049 INFO:teuthology.orchestra.run.smithi165.stdout:3418 clone 28 order 19 su 65536 sc 2 2024-09-17T21:47:21.076 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:21.073+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:21.109 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:21.109 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:21.109 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:21.109 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:21.109 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:21.118 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:21.176 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:21.174+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4189.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e14)", 2024-09-17T21:47:21.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 94.064572213000005, 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.098108624000000005, 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4189", 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2976419226", 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158802+0000", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158803+0000", 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5e-06 2024-09-17T21:47:21.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158812+0000", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6030000000000005e-06 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158822+0000", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0685e-05 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158833+0000", 2024-09-17T21:47:21.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0335e-05 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158904+0000", 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1413000000000002e-05 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.158911+0000", 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2219999999999996e-06 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255566+0000", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.096654411999999995 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255692+0000", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000126155 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.255876+0000", 2024-09-17T21:47:21.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018391999999999999 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256582+0000", 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070622700000000003 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256838+0000", 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000255732 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256878+0000", 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0054e-05 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:45:47.256910+0000", 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2366000000000001e-05 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:914 3.12 3:4ea287c8:::rbd_data.1112a8bc19be.000000000000000f:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.403586924000003, 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026647334000000002, 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 914 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819787+0000", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0100000000000005e-07 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819791+0000", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1039999999999998e-06 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819798+0000", 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3459999999999996e-06 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819804+0000", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.819857+0000", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3297999999999997e-05 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.820073+0000", 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021552600000000001 2024-09-17T21:47:21.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.846434+0000", 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026361286000000001 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2117 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 425984~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:21.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.386015532999998, 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019370693000000001, 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2117 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:21.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837358+0000", 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837359+0000", 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4999999999999998e-07 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837371+0000", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2012e-05 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837378+0000", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0829999999999998e-06 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837383+0000", 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2610000000000001e-06 2024-09-17T21:47:21.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856530+0000", 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019147352999999999 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856627+0000", 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6775999999999996e-05 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856729+0000", 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101758 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2125 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 434176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.385981839999999, 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019564181999999999, 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2125 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837392+0000", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837394+0000", 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837401+0000", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1340000000000001e-06 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.837406+0000", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856757+0000", 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019351133 2024-09-17T21:47:21.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856847+0000", 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0420000000000005e-05 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.856956+0000", 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010900699999999999 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2221 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 516096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.383140537000003, 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019220732000000001, 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2221 2024-09-17T21:47:21.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840233+0000", 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840234+0000", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840236+0000", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.013e-06 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840242+0000", 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3219999999999998e-06 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840248+0000", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859296+0000", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019048430000000002 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859369+0000", 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2829000000000004e-05 2024-09-17T21:47:21.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859454+0000", 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5107999999999999e-05 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2241 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.383063225999997, 2024-09-17T21:47:21.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019328267999999999, 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2241 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840311+0000", 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2e-07 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840313+0000", 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840321+0000", 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.131e-06 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840326+0000", 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2050000000000001e-06 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859483+0000", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019157045000000001 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859555+0000", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2075000000000004e-05 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859639+0000", 2024-09-17T21:47:21.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3938999999999997e-05 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2245 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 745472~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.383040833000003, 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019500021999999999, 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2245 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:21.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840333+0000", 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840335+0000", 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.04e-06 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840342+0000", 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6390000000000003e-06 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840347+0000", 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2800000000000003e-06 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859665+0000", 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019317889000000001 2024-09-17T21:47:21.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859738+0000", 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2929999999999995e-05 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859833+0000", 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4716000000000006e-05 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2248 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 458752~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.383019711999999, 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019678878, 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2248 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840354+0000", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840356+0000", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9319999999999999e-06 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840363+0000", 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840369+0000", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3120000000000001e-06 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859860+0000", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019491867999999999 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.859932+0000", 2024-09-17T21:47:21.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1675999999999996e-05 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860033+0000", 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000100859 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2251 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 466944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.382998968000003, 2024-09-17T21:47:21.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019911997000000001, 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2251 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840375+0000", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7399999999999998e-07 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840377+0000", 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:47:21.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840384+0000", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840389+0000", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860076+0000", 2024-09-17T21:47:21.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019687455999999999 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860177+0000", 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101347 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860287+0000", 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109332 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2271 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 688128~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.382584324, 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019723041, 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2271 2024-09-17T21:47:21.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840789+0000", 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840790+0000", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3900000000000003e-07 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840795+0000", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840806+0000", 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1112e-05 2024-09-17T21:47:21.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840813+0000", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3309999999999998e-06 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860322+0000", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019508976000000001 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860411+0000", 2024-09-17T21:47:21.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8838999999999994e-05 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860512+0000", 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010135400000000001 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2275 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:21.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.382549132999998, 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019949587000000001, 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2275 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840825+0000", 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840828+0000", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5849999999999998e-06 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840835+0000", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4039999999999998e-06 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.840843+0000", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.576e-06 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860547+0000", 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019704342999999999 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860634+0000", 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7281999999999995e-05 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860774+0000", 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000139887 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2291 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 966656~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.381657670000003, 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019282127999999999, 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2291 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841716+0000", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841717+0000", 2024-09-17T21:47:21.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.47e-07 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841721+0000", 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4730000000000002e-06 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841731+0000", 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.841738+0000", 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0770000000000002e-06 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860805+0000", 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019066512000000001 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860888+0000", 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3162000000000001e-05 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.860998+0000", 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000110287 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:2293 3.d 3:b2683f28:::rbd_data.1112e01b91a8.0000000000000000:9 [read 974848~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.381291601000001, 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019306116000000002, 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2293 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842082+0000", 2024-09-17T21:47:21.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842083+0000", 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.003e-06 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842088+0000", 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3719999999999998e-06 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842098+0000", 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0001e-05 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.842105+0000", 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4359999999999996e-06 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861026+0000", 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018920929999999999 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861162+0000", 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013602500000000001 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.861388+0000", 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022634900000000001 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3955 3.d 3:b74dd9cd:::rbd_data.11121f8e87bb.0000000000000010:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.327984383, 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019321438, 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3955 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895389+0000", 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895390+0000", 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895402+0000", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1960000000000001e-05 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895409+0000", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895417+0000", 2024-09-17T21:47:21.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914537+0000", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019119367000000002 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914612+0000", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5665000000000001e-05 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914711+0000", 2024-09-17T21:47:21.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8443000000000006e-05 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:3956 3.15 3:ae039d79:::rbd_data.11121f8e87bb.0000000000000011:8 [read 77824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.327948610999997, 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019259279000000001, 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3956 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895425+0000", 2024-09-17T21:47:21.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895426+0000", 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895428+0000", 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895434+0000", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2689999999999996e-06 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.895439+0000", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914517+0000", 2024-09-17T21:47:21.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019077696000000002 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914588+0000", 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1667000000000003e-05 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.914684+0000", 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6119000000000006e-05 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4084 3.15 3:ae6012c0:::rbd_data.1112a8bc19be.0000000000000014:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.323810238999997, 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026954235999999999, 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4084 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899563+0000", 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899564+0000", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8700000000000002e-07 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899566+0000", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.818e-06 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899570+0000", 2024-09-17T21:47:21.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9450000000000003e-06 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.899572+0000", 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7250000000000002e-06 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915650+0000", 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016077382000000001 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915719+0000", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9622999999999994e-05 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.915912+0000", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019283000000000001 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922420+0000", 2024-09-17T21:47:21.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065078499999999999 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926421+0000", 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040006590000000002 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926475+0000", 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4338000000000001e-05 2024-09-17T21:47:21.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926518+0000", 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2778999999999998e-05 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4720.0:4120 3.15 3:aeafc417:::rbd_data.1112a8bc19be.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.318616517999999, 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021859593, 2024-09-17T21:47:21.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4720", 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4120 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:21.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904757+0000", 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.904758+0000", 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909494+0000", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047359960000000001 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909505+0000", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1406999999999999e-05 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.909513+0000", 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610000000000003e-06 2024-09-17T21:47:21.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916227+0000", 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067142909999999998 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916299+0000", 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1703000000000001e-05 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:21.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.916705+0000", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000406289 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.922456+0000", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0057514330000000002 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926527+0000", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0040706630000000004 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926577+0000", 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9731000000000002e-05 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:31.926617+0000", 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0159e-05 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4673.0:711 3.d 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 28327424~10752] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e50)", 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:21.311 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 46.926109924999999, 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.027994687000000001, 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4673", 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 711 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297264+0000", 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.297265+0000", 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:47:21.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305091+0000", 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0078258979999999995 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305109+0000", 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8643000000000001e-05 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305117+0000", 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2879999999999993e-06 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305180+0000", 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2967999999999997e-05 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305293+0000", 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112484 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.305860+0000", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00056668599999999995 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.317427+0000", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011567409000000001 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325191+0000", 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.007763926 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325229+0000", 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.786e-05 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:34.325258+0000", 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9689999999999999e-05 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.317 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:364 3.14 3:2c59c287:::rbd_data.111223014882.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 13130752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.354810192000002, 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022145847999999999, 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 364 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868564+0000", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.868566+0000", 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4480000000000001e-06 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874073+0000", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055071290000000004 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874087+0000", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.379e-05 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874093+0000", 2024-09-17T21:47:21.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874145+0000", 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2454999999999998e-05 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874290+0000", 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144282 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:21.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874720+0000", 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043065399999999999 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.886603+0000", 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011882805999999999 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890574+0000", 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039709100000000002 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890642+0000", 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8017999999999998e-05 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.890709+0000", 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7321000000000001e-05 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4822.0:366 3.11 3:8c7f8f26:::rbd_data.111223014882.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 12582912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.349262828999997, 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.026323974, 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4822", 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/3108341854", 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 366 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874111+0000", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.874112+0000", 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8100000000000001e-07 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883245+0000", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0091332389999999996 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883262+0000", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.698e-05 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883274+0000", 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1568e-05 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883344+0000", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0457999999999995e-05 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883456+0000", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111844 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.883889+0000", 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043312599999999999 2024-09-17T21:47:21.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900177+0000", 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016288140999999999 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900339+0000", 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016148700000000001 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900390+0000", 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1146999999999997e-05 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:35.900435+0000", 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5003000000000003e-05 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.329 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:21.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:21.331 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:21.328+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:21.431 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:21.429+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:21.479 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:21.479 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:21.479 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:21.479 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:21.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:21.532 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:21.530+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:21.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:21.633 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:21.630+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:21.634 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:21.634 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:21.635 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:21.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:21.734 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:21.731+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:21.751 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:21.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:21.768 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:21.784 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:21.784 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:21.793 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:21.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.838984121999999, 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:21.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.836959070000006, 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:21.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:21.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.822973013999999, 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.821011807000005, 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.818978450000003, 2024-09-17T21:47:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.817000531999994, 2024-09-17T21:47:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.816977738000006, 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.811068675000001, 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.809037615999998, 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.809011041999995, 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.808401248999999, 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.808383663000001, 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.808368705999996, 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.092687217999995, 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.092656597000001, 2024-09-17T21:47:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.092631979000004, 2024-09-17T21:47:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.092610031999996, 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.092588562000003, 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.051171758999999, 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.050127299000003, 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:21.901 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:21.903 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:21.903 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:21.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:21.905 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:21.899+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:21.927 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:47:22.002 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone27 from 0xeb3e400 (overlap 0x8e8200) to 0x1c8000 2024-09-17T21:47:22.003 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:22.000+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:22.067 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.067 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:22.119 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:22.117+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:22.183 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.183 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.191 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:22.205 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:22.203+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:22.306 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:22.303+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:22.338 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.338 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:22.407 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:22.404+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:22.457 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.457 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.466 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:22.510 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:22.508+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.573 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:22.608 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:22.605+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:22.709 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:22.707+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:22.729 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.729 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:22.810 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:22.808+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:22.885 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:22.886 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:22.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:22.911 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:22.908+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.004 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.012 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:23.010+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:23.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:23.113 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:23.111+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:23.128 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.128 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:23.214 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:23.212+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:23.255 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.255 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:23.316 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:23.313+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:23.375 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.376 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:23.406 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-17T21:47:23.430 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:23.428+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:23.507 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.507 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.513 INFO:teuthology.orchestra.run.smithi165.stdout:3422 punch from 0xa71e800 to 0xa726c00, (0x8400 bytes) 2024-09-17T21:47:23.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:47:23.517 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:23.515+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:23.521 INFO:teuthology.orchestra.run.smithi165.stdout:3423 punch from 0x7b31a00 to 0x7b41800, (0xfe00 bytes) 2024-09-17T21:47:23.540 INFO:teuthology.orchestra.run.smithi165.stdout:3425 trunc from 0xeb3e400 to 0x268e800 2024-09-17T21:47:23.558 INFO:teuthology.orchestra.run.smithi165.stdout:3426 punch from 0x1e3da00 to 0x1e45600, (0x7c00 bytes) 2024-09-17T21:47:23.559 INFO:teuthology.orchestra.run.smithi165.stdout:3428 read 0x2151a00 thru 0x215bbaf (0xa1b0 bytes) 2024-09-17T21:47:23.559 INFO:teuthology.orchestra.run.smithi165.stdout:3430 read 0x1eebe00 thru 0x1ef735a (0xb55b bytes) 2024-09-17T21:47:23.585 INFO:teuthology.orchestra.run.smithi165.stdout:3431 write 0xec9f000 thru 0xecacf1a (0xdf1b bytes) 2024-09-17T21:47:23.599 INFO:teuthology.orchestra.run.smithi165.stdout:3432 trunc from 0xecad000 to 0x1705000 2024-09-17T21:47:23.619 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:23.616+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:23.633 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.633 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.634 INFO:teuthology.orchestra.run.smithi165.stdout:3433 write 0x598da00 thru 0x5999cd4 (0xc2d5 bytes) 2024-09-17T21:47:23.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:47:23.661 INFO:teuthology.orchestra.run.smithi165.stdout:3434 write 0xec55a00 thru 0xec579b4 (0x1fb5 bytes) 2024-09-17T21:47:23.679 INFO:teuthology.orchestra.run.smithi165.stdout:3435 trunc from 0xec57a00 to 0x9e8400 2024-09-17T21:47:23.719 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:23.717+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:23.723 INFO:teuthology.orchestra.run.smithi165.stdout:3436 write 0xb9f8a00 thru 0xb9fad9f (0x23a0 bytes) 2024-09-17T21:47:23.725 INFO:teuthology.orchestra.run.smithi165.stdout:3438 trunc from 0xb9fae00 to 0xa924200 2024-09-17T21:47:23.740 INFO:teuthology.orchestra.run.smithi165.stdout:3439 read 0x7c1e400 thru 0x7c2de0a (0xfa0b bytes) 2024-09-17T21:47:23.741 INFO:teuthology.orchestra.run.smithi165.stdout:3442 read 0x6cc8800 thru 0x6ccecc5 (0x64c6 bytes) 2024-09-17T21:47:23.743 INFO:teuthology.orchestra.run.smithi165.stdout:3446 trunc from 0xa924200 to 0x8fc0e00 2024-09-17T21:47:23.753 INFO:teuthology.orchestra.run.smithi165.stdout:3447 punch from 0x64a3a00 to 0x64a8400, (0x4a00 bytes) 2024-09-17T21:47:23.754 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.754 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.756 INFO:teuthology.orchestra.run.smithi165.stdout:3448 trunc from 0x8fc0e00 to 0x7507400 2024-09-17T21:47:23.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:47:23.767 INFO:teuthology.orchestra.run.smithi165.stdout:3449 punch from 0x4826a00 to 0x4827e00, (0x1400 bytes) 2024-09-17T21:47:23.789 INFO:teuthology.orchestra.run.smithi165.stdout:3451 write 0xed29a00 thru 0xed2c738 (0x2d39 bytes) 2024-09-17T21:47:23.790 INFO:teuthology.orchestra.run.smithi165.stdout:3452 write 0x78f7400 thru 0x7903eb3 (0xcab4 bytes) 2024-09-17T21:47:23.790 INFO:teuthology.orchestra.run.smithi165.stdout:3455 punch from 0x252f400 to 0x253cc00, (0xd800 bytes) 2024-09-17T21:47:23.790 INFO:teuthology.orchestra.run.smithi165.stdout:3458 read 0x233b000 thru 0x233b498 (0x499 bytes) 2024-09-17T21:47:23.790 INFO:teuthology.orchestra.run.smithi165.stdout:3459 read 0xa76400 thru 0xa80fad (0xabae bytes) 2024-09-17T21:47:23.800 INFO:teuthology.orchestra.run.smithi165.stdout:3460 trunc from 0xed2c800 to 0x3ae0c00 2024-09-17T21:47:23.816 INFO:teuthology.orchestra.run.smithi165.stdout:3461 read 0x91200 thru 0x9d677 (0xc478 bytes) 2024-09-17T21:47:23.833 INFO:teuthology.orchestra.run.smithi165.stdout:3462 write 0x77c3a00 thru 0x77cbd57 (0x8358 bytes) 2024-09-17T21:47:23.834 INFO:teuthology.orchestra.run.smithi165.stdout:3464 write 0x425fe00 thru 0x4260e87 (0x1088 bytes) 2024-09-17T21:47:23.834 INFO:teuthology.orchestra.run.smithi165.stdout:3465 read 0x6f0d800 thru 0x6f1857d (0xad7e bytes) 2024-09-17T21:47:23.834 INFO:teuthology.orchestra.run.smithi165.stdout:3466 trunc from 0x77cbe00 to 0x7b8d000 2024-09-17T21:47:23.847 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:23.845+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:23.848 INFO:teuthology.orchestra.run.smithi165.stdout:3468 trunc from 0x7b8d000 to 0x6d90800 2024-09-17T21:47:23.870 INFO:teuthology.orchestra.run.smithi165.stdout:3471 trunc from 0x6d90800 to 0xa4a9c00 2024-09-17T21:47:23.903 INFO:teuthology.orchestra.run.smithi165.stdout:3477 write 0xea6d800 thru 0xea70889 (0x308a bytes) 2024-09-17T21:47:23.903 INFO:teuthology.orchestra.run.smithi165.stdout:3478 read 0x459c800 thru 0x45a3121 (0x6922 bytes) 2024-09-17T21:47:23.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:23.914 INFO:teuthology.orchestra.run.smithi165.stdout:3483 trunc from 0xea70a00 to 0x4f7ca00 2024-09-17T21:47:23.915 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:23.916 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:23.921 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:23.918+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:23.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:23.934 INFO:teuthology.orchestra.run.smithi165.stdout:3485 write 0xd8b000 thru 0xd98b53 (0xdb54 bytes) 2024-09-17T21:47:23.935 INFO:teuthology.orchestra.run.smithi165.stdout:3487 read 0x253a00 thru 0x263044 (0xf645 bytes) 2024-09-17T21:47:23.936 INFO:teuthology.orchestra.run.smithi165.stdout:3493 write 0x1bb4e00 thru 0x1bbd9ff (0x8c00 bytes) 2024-09-17T21:47:23.936 INFO:teuthology.orchestra.run.smithi165.stdout:3494 read 0x153e600 thru 0x1549456 (0xae57 bytes) 2024-09-17T21:47:23.936 INFO:teuthology.orchestra.run.smithi165.stdout:3495 read 0x40b2400 thru 0x40c1ea9 (0xfaaa bytes) 2024-09-17T21:47:23.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:47:23.959 INFO:teuthology.orchestra.run.smithi165.stdout:3499 write 0xbbdb200 thru 0xbbe1d38 (0x6b39 bytes) 2024-09-17T21:47:23.960 INFO:teuthology.orchestra.run.smithi165.stdout:3500 write 0x69cf400 thru 0x69d73ed (0x7fee bytes) 2024-09-17T21:47:23.960 INFO:teuthology.orchestra.run.smithi165.stdout:3501 write 0x5532c00 thru 0x553e673 (0xba74 bytes) 2024-09-17T21:47:23.960 INFO:teuthology.orchestra.run.smithi165.stdout:3506 read 0x5865e00 thru 0x5870699 (0xa89a bytes) 2024-09-17T21:47:23.966 INFO:teuthology.orchestra.run.smithi165.stdout:3507 trunc from 0xbbe1e00 to 0x6f5c200 2024-09-17T21:47:23.992 INFO:teuthology.orchestra.run.smithi165.stdout:3509 read 0x4d54400 thru 0x4d5714b (0x2d4c bytes) 2024-09-17T21:47:23.995 INFO:teuthology.orchestra.run.smithi165.stdout:3510 write 0x4721200 thru 0x4724fe1 (0x3de2 bytes) 2024-09-17T21:47:24.014 INFO:teuthology.orchestra.run.smithi165.stdout:3511 write 0xb29be00 thru 0xb2a3775 (0x7976 bytes) 2024-09-17T21:47:24.021 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:24.019+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:24.034 INFO:teuthology.orchestra.run.smithi165.stdout:3516 write 0xbaf7600 thru 0xbb011ad (0x9bae bytes) 2024-09-17T21:47:24.040 INFO:teuthology.orchestra.run.smithi165.stdout:3519 write 0x6a04a00 thru 0x6a06486 (0x1a87 bytes) 2024-09-17T21:47:24.040 INFO:teuthology.orchestra.run.smithi165.stdout:3520 write 0x1bb0c00 thru 0x1bb2652 (0x1a53 bytes) 2024-09-17T21:47:24.040 INFO:teuthology.orchestra.run.smithi165.stdout:3522 punch from 0xada2c00 to 0xada9a00, (0x6e00 bytes) 2024-09-17T21:47:24.047 INFO:teuthology.orchestra.run.smithi165.stdout:3525 trunc from 0xbb01200 to 0x6af9800 2024-09-17T21:47:24.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:24.065 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:24.066 INFO:teuthology.orchestra.run.smithi165.stdout:3527 trunc from 0x6af9800 to 0x5489800 2024-09-17T21:47:24.079 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:24.079 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:24.089 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:24.089 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:24.090 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:24.098 INFO:teuthology.orchestra.run.smithi165.stdout:3533 trunc from 0x5489800 to 0xdd70a00 2024-09-17T21:47:24.108 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:24.108 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:24.115 INFO:teuthology.orchestra.run.smithi165.stdout:3534 read 0x586f000 thru 0x586f08c (0x8d bytes) 2024-09-17T21:47:24.128 INFO:teuthology.orchestra.run.smithi165.stdout:3535 trunc from 0xdd70a00 to 0x3491a00 2024-09-17T21:47:24.130 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:24.128+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:24.141 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:47:24.141 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:47:24.146 INFO:teuthology.orchestra.run.smithi165.stdout:3539 write 0x33d1000 thru 0x33d432b (0x332c bytes) 2024-09-17T21:47:24.146 INFO:teuthology.orchestra.run.smithi165.stdout:3541 write 0x794800 thru 0x7a3f4e (0xf74f bytes) 2024-09-17T21:47:24.147 INFO:teuthology.orchestra.run.smithi165.stdout:3543 read 0x15d3400 thru 0x15d7cb0 (0x48b1 bytes) 2024-09-17T21:47:24.147 INFO:teuthology.orchestra.run.smithi165.stdout:3544 read 0xc90000 thru 0xc96107 (0x6108 bytes) 2024-09-17T21:47:24.147 INFO:teuthology.orchestra.run.smithi165.stdout:3546 write 0x10dd200 thru 0x10e60ab (0x8eac bytes) 2024-09-17T21:47:24.147 INFO:teuthology.orchestra.run.smithi165.stdout:3550 read 0x8aee00 thru 0x8b1942 (0x2b43 bytes) 2024-09-17T21:47:24.161 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:24.161 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:24.173 INFO:teuthology.orchestra.run.smithi165.stdout:3552 write 0xaa4f800 thru 0xaa4fe4f (0x650 bytes) 2024-09-17T21:47:24.174 INFO:teuthology.orchestra.run.smithi165.stdout:3553 trunc from 0xaa50000 to 0xa9c4e00 2024-09-17T21:47:24.194 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:24.194 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:24.196 INFO:teuthology.orchestra.run.smithi165.stdout:3555 read 0x9a30c00 thru 0x9a34321 (0x3722 bytes) 2024-09-17T21:47:24.196 INFO:teuthology.orchestra.run.smithi165.stdout:3557 read 0x95bf800 thru 0x95c8646 (0x8e47 bytes) 2024-09-17T21:47:24.196 INFO:teuthology.orchestra.run.smithi165.stdout:3559 write 0x824ea00 thru 0x82580e8 (0x96e9 bytes) 2024-09-17T21:47:24.209 INFO:teuthology.orchestra.run.smithi165.stdout:3560 trunc from 0xa9c4e00 to 0x62ea00 2024-09-17T21:47:24.223 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:24.220+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:24.243 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:24.243 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:24.243 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:24.243 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:24.244 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:24.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:24.257 INFO:teuthology.orchestra.run.smithi165.stdout:3561 write 0x9bcba00 thru 0x9bd67ff (0xae00 bytes) 2024-09-17T21:47:24.258 INFO:teuthology.orchestra.run.smithi165.stdout:3562 trunc from 0x9bd6800 to 0x93fdc00 2024-09-17T21:47:24.277 INFO:teuthology.orchestra.run.smithi165.stdout:3564 write 0x6d30600 thru 0x6d3955d (0x8f5e bytes) 2024-09-17T21:47:24.291 INFO:teuthology.orchestra.run.smithi165.stdout:3566 write 0x98de800 thru 0x98e5b71 (0x7372 bytes) 2024-09-17T21:47:24.291 INFO:teuthology.orchestra.run.smithi165.stdout:3572 clone 29 order 23 su 4194304 sc 12 2024-09-17T21:47:24.336 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:24.334+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.447453953999997, 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:24.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:24.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.445428902000003, 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:24.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.431442845999996, 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:24.412 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:24.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:24.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.429481639000002, 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:24.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:24.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.427448282, 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:24.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.425470364000006, 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:24.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:24.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.425447570000003, 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.419538506999999, 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:24.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.417507447999995, 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:24.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.417480874000006, 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:24.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.451 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.416871080999996, 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:24.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:24.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:24.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.416853494999998, 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:24.457 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:24.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.416838537999993, 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:24.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.701157050000006, 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.474 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.701126428999999, 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:24.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:24.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.701101811000001, 2024-09-17T21:47:24.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:24.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.701079863999993, 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.486 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:24.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.491 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.701058394, 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:24.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:24.497 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.659641591000003, 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:24.498 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:24.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:24.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.658597131000001, 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:24.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:24.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.508 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:24.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:24.510 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:24.507+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:24.610 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:24.608+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:24.656 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:47:24.657 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:24.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:24.658 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:24.658 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:24.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:24.710 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:24.708+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:24.731 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone28 from 0x98e5c00 (overlap 0x62ea00) to 0x0 2024-09-17T21:47:24.811 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:24.809+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:24.822 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:24.822 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:24.822 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:24.822 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:24.822 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:24.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:24.912 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:24.910+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:24.941 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967650243999998, 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967627750999995, 2024-09-17T21:47:24.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:24.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:24.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967603491000006, 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.951 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:24.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:24.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967577607999999, 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967552607000002, 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.959 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:24.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:24.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967512807999995, 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:24.963 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:24.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:24.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.967 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967497062999996, 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967485527999997, 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:24.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:24.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.967475227999998, 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.976 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:24.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965503984999998, 2024-09-17T21:47:24.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:24.984 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965490837000004, 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:24.988 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965480231000001, 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:24.992 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965469893000005, 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965459886000005, 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:24.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:25.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965449934999995, 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:25.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:25.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.965434626999993, 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.005 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:25.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.963458739999993, 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:25.009 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:25.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.963447783999996, 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:25.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.963435294999996, 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.018 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:25.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:25.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:25.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.963424832000001, 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.022 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:25.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:25.026 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:25.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:25.028 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:25.025+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:25.049 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-17T21:47:25.120 INFO:teuthology.orchestra.run.smithi165.stdout:3573 read 0x5b87a00 thru 0x5b88c95 (0x1296 bytes) 2024-09-17T21:47:25.120 INFO:teuthology.orchestra.run.smithi165.stdout:3577 read 0x82aa600 thru 0x82b2f4f (0x8950 bytes) 2024-09-17T21:47:25.121 INFO:teuthology.orchestra.run.smithi165.stdout:3578 write 0x3d20800 thru 0x3d2bdf7 (0xb5f8 bytes) 2024-09-17T21:47:25.128 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:25.126+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:25.136 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:25.136 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:25.153 INFO:teuthology.orchestra.run.smithi165.stdout:3579 write 0x1c5ba00 thru 0x1c60155 (0x4756 bytes) 2024-09-17T21:47:25.154 INFO:teuthology.orchestra.run.smithi165.stdout:3580 punch from 0x8a23000 to 0x8a2da00, (0xaa00 bytes) 2024-09-17T21:47:25.165 INFO:teuthology.orchestra.run.smithi165.stdout:3581 write 0x5cd1c00 thru 0x5cd2076 (0x477 bytes) 2024-09-17T21:47:25.165 INFO:teuthology.orchestra.run.smithi165.stdout:3582 punch from 0x9601200 to 0x960d200, (0xc000 bytes) 2024-09-17T21:47:25.171 INFO:teuthology.orchestra.run.smithi165.stdout:3585 read 0x3f0d800 thru 0x3f17c62 (0xa463 bytes) 2024-09-17T21:47:25.174 INFO:teuthology.orchestra.run.smithi165.stdout:3591 trunc from 0x98e5c00 to 0xb2bd400 2024-09-17T21:47:25.184 INFO:teuthology.orchestra.run.smithi165.stdout:3595 read 0x63f6e00 thru 0x6403a79 (0xcc7a bytes) 2024-09-17T21:47:25.189 INFO:teuthology.orchestra.run.smithi165.stdout:3597 punch from 0xab88000 to 0xab97000, (0xf000 bytes) 2024-09-17T21:47:25.189 INFO:teuthology.orchestra.run.smithi165.stdout:3600 read 0x651c800 thru 0x6522363 (0x5b64 bytes) 2024-09-17T21:47:25.202 INFO:teuthology.orchestra.run.smithi165.stdout:3601 write 0xc55ae00 thru 0xc56465b (0x985c bytes) 2024-09-17T21:47:25.203 INFO:teuthology.orchestra.run.smithi165.stdout:3602 punch from 0x8db000 to 0x8e2200, (0x7200 bytes) 2024-09-17T21:47:25.204 INFO:teuthology.orchestra.run.smithi165.stdout:3603 read 0xae8d000 thru 0xae8fc7e (0x2c7f bytes) 2024-09-17T21:47:25.204 INFO:teuthology.orchestra.run.smithi165.stdout:3604 punch from 0x4463a00 to 0x446f000, (0xb600 bytes) 2024-09-17T21:47:25.204 INFO:teuthology.orchestra.run.smithi165.stdout:3605 write 0x8578e00 thru 0x858586d (0xca6e bytes) 2024-09-17T21:47:25.204 INFO:teuthology.orchestra.run.smithi165.stdout:3608 punch from 0x21aac00 to 0x21ba000, (0xf400 bytes) 2024-09-17T21:47:25.207 INFO:teuthology.orchestra.run.smithi165.stdout:3610 trunc from 0xc564800 to 0xe673400 2024-09-17T21:47:25.219 INFO:teuthology.orchestra.run.smithi165.stdout:3611 write 0x295be00 thru 0x296243d (0x663e bytes) 2024-09-17T21:47:25.220 INFO:teuthology.orchestra.run.smithi165.stdout:3613 read 0xb43b800 thru 0xb43dfdc (0x27dd bytes) 2024-09-17T21:47:25.221 INFO:teuthology.orchestra.run.smithi165.stdout:3615 write 0xc4c5c00 thru 0xc4d02a5 (0xa6a6 bytes) 2024-09-17T21:47:25.221 INFO:teuthology.orchestra.run.smithi165.stdout:3617 clone 30 order 25 su 2097152 sc 6 2024-09-17T21:47:25.229 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:47:25.227+0000 7f43ef229640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32303) UID: 0 2024-09-17T21:47:25.258 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:25.258 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:25.259 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:25.330 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:25.328+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:25.376 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.376 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.690359897, 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:25.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:25.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.658084922, 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:25.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:25.388 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.657846153999998, 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:25.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.513136539000001, 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:25.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:627 3.6 3:6244dd87:::rbd_data.1112bc1b2568.0000000000000019:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.422616806000001, 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030691610000000001, 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 627 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950565+0000", 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:25.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950567+0000", 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950574+0000", 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950580+0000", 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950631+0000", 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1863999999999997e-05 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950752+0000", 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120596 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981256+0000", 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030503834000000001 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:745 3.6 3:60bbb74c:::rbd_data.1112bc1b2568.00000000000000de:head [delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.414707780000001, 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023010439000000001, 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 745 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958474+0000", 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958476+0000", 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2299999999999998e-06 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958482+0000", 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958488+0000", 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961234+0000", 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027465929999999999 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961306+0000", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1521999999999997e-05 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961496+0000", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190053 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.962386+0000", 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089013799999999995 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981403+0000", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019016943000000001 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981448+0000", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5212e-05 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981484+0000", 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5608999999999997e-05 2024-09-17T21:47:25.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1235 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.386151256000002, 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013460730000000001, 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1235 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987032+0000", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9209999999999999e-06 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987043+0000", 2024-09-17T21:47:25.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1076e-05 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987049+0000", 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000331+0000", 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013281866999999999 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000405+0000", 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4004000000000005e-05 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000491+0000", 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6062000000000004e-05 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1236 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.386125326999998, 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887803000000001, 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1236 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:47:25.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987058+0000", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987065+0000", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1219999999999997e-06 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987070+0000", 2024-09-17T21:47:25.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9529999999999998e-06 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000520+0000", 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013449796999999999 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000592+0000", 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919e-05 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000944+0000", 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035145700000000001 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1237 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.38610422, 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049556, 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1237 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987080+0000", 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:25.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987086+0000", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6309999999999997e-06 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987091+0000", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000974+0000", 2024-09-17T21:47:25.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013882864999999999 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001045+0000", 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0661000000000007e-05 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001126+0000", 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1521999999999996e-05 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1264 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.385933300000001, 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014065061, 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1264 2024-09-17T21:47:25.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987249+0000", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987252+0000", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987263+0000", 2024-09-17T21:47:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1289e-05 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987270+0000", 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1330000000000001e-06 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001154+0000", 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01388413 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001225+0000", 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0975000000000004e-05 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001313+0000", 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7404999999999994e-05 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1265 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:25.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.385899989999999, 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01421443, 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1265 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987282+0000", 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987284+0000", 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.328e-06 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987291+0000", 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:47:25.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987305+0000", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3344e-05 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001341+0000", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014035903000000001 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001411+0000", 2024-09-17T21:47:25.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0502999999999999e-05 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001496+0000", 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4383999999999997e-05 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1266 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.385868049999999, 2024-09-17T21:47:25.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014670829, 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1266 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987314+0000", 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987316+0000", 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4059999999999999e-06 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987323+0000", 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987334+0000", 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.093e-05 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001524+0000", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014189715 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001592+0000", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8132000000000004e-05 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001984+0000", 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039185199999999999 2024-09-17T21:47:25.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1267 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.38583908, 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014824201, 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1267 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987343+0000", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987345+0000", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987351+0000", 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:47:25.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987356+0000", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002014+0000", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014657411 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002084+0000", 2024-09-17T21:47:25.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0596e-05 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002166+0000", 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2083999999999995e-05 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1268 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.385817431, 2024-09-17T21:47:25.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014992027999999999, 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1268 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987369+0000", 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7129999999999996e-06 2024-09-17T21:47:25.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987376+0000", 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987381+0000", 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002195+0000", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014814226 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002268+0000", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3279000000000001e-05 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002356+0000", 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7305999999999994e-05 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1269 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.385790267000001, 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015291325999999999, 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1269 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987394+0000", 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3989999999999998e-06 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987401+0000", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8210000000000004e-06 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987406+0000", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0030000000000002e-06 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002522+0000", 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015116368 2024-09-17T21:47:25.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002595+0000", 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3335000000000002e-05 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002682+0000", 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6927000000000003e-05 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:771 3.1 3:8006aab1:::rbd_data.1112f0bef54a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 8280576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.769037214000001, 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013777717, 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 771 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604145+0000", 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T21:47:25.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608094+0000", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039492950000000002 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608114+0000", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0254e-05 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608122+0000", 2024-09-17T21:47:25.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608205+0000", 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3511999999999996e-05 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608304+0000", 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8524999999999996e-05 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:25.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608621+0000", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031733999999999999 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617430+0000", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088086459999999998 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617787+0000", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035710100000000002 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:25.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617854+0000", 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7699000000000004e-05 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617922+0000", 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7169999999999996e-05 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.490340024, 2024-09-17T21:47:25.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:25.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.483438252999999, 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:25.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.483322162, 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:25.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2165 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch ping cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.177087083, 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69677434100000002, 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2165 2024-09-17T21:47:25.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196096+0000", 2024-09-17T21:47:25.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196108+0000", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196130+0000", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1695000000000002e-05 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196140+0000", 2024-09-17T21:47:25.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196216+0000", 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899e-05 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196227+0000", 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1008e-05 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892718+0000", 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69649070000000002 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892759+0000", 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1876999999999999e-05 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892868+0000", 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010892599999999999 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.477 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.478 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:25.479 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:25.475+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:25.579 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:25.576+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:25.586 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:25.586 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:25.680 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:25.677+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:25.741 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:25.741 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:25.741 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:25.742 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:25.780 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:25.778+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:25.858 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:25.858 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951613932000001, 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.859 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:25.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:25.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:25.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951603630999998, 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951587556999996, 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951576344000003, 2024-09-17T21:47:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951558906000002, 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951548036000005, 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.951353159000007, 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:25.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.949648488999998, 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:25.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:25.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.949637851000006, 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:25.892 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.949627758000005, 2024-09-17T21:47:25.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:25.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.949617638999996, 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:25.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:25.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.947487409000004, 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.947477677999998, 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:25.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:25.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.946694319000002, 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:25.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.946678749, 2024-09-17T21:47:25.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:25.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:25.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.946585846999994, 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:25.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:25.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.946091228, 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.927 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.945947183000001, 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:25.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.945261746, 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.944899355999993, 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:25.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:25.945 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:25.945 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5] 2024-09-17T21:47:25.945 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:25.945 INFO:tasks.thrashosds.thrasher:Killing osd 3, live_osds are [0, 1, 2, 3, 4, 5] 2024-09-17T21:47:25.945 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T21:47:25.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:47:25.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:25.947 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:25.943+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:25.982 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T21:47:25.982 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.3 2024-09-17T21:47:25.982 INFO:tasks.thrashosds.thrasher:remote for osd 3 is ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:47:25.982 DEBUG:teuthology.orchestra.run.smithi053:> type ceph-objectstore-tool 2024-09-17T21:47:26.039 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-17T21:47:26.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op list-pgs 2024-09-17T21:47:26.080 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:26.078+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:26.096 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:26.097 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:26.097 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:26.097 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:26.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:26.149 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:26.147+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:26.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:26.213 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:26.213 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:26.213 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:26.214 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:26.214 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:26.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:26.248 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:26.246+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:26.332 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:26.332 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:26.349 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:26.347+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:26.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:26.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:26.366 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:26.367 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:26.373 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:26.373 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:26.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:26.449 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:26.447+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:26.500 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:26.500 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:26.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:26.550 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:26.547+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:26.650 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:26.648+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:26.656 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:26.656 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:26.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:26.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:47:26.751 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:26.749+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.777 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:10993 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 479232~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.822600281000007, 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065241656999999995, 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10993 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.951157+0000", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952946+0000", 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001788581 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952950+0000", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1119999999999998e-06 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952961+0000", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0879999999999999e-05 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.952968+0000", 2024-09-17T21:47:26.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5970000000000003e-06 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016204+0000", 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063236106 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016252+0000", 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-05 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.016399+0000", 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014719099999999999 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.820575228999999, 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.806589173000006, 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:26.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.804627965999998, 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.802594608999996, 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:26.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.800616691000002, 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:26.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:26.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.800593896999999, 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:26.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:26.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:26.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.794684833999995, 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:26.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:26.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.792653775000005, 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:26.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:26.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:26.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.792627201000002, 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:26.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:26.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.792017408000007, 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:26.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.791999821999994, 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.830 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:26.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.791984865000003, 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:26.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:26.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:26.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:26.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:26.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.076303377000002, 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:26.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:26.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.076272755999995, 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:26.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:26.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.076248137999997, 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:26.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:26.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:26.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.076226191000003, 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:26.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:26.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:26.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.076204720999996, 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:26.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:26.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.034787917999999, 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:26.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.033743457999996, 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:26.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:26.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:26.882 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:26.884 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:26.884 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:26.881+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:26.984 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:26.982+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:27.040 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:27.040 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:27.040 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:27.040 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:27.049 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:27.085 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:27.083+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:27.160 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:27.160 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:27.161 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:27.161 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:27.161 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:27.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:27.213 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:27.211+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306924707999997, 2024-09-17T21:47:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306902214999994, 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306877955000004, 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306852071999998, 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306827071000001, 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:27.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306787271999994, 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306771526999995, 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:27.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306759991999996, 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:27.313 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:27.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:27.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:27.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.306749691999997, 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:27.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:27.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304778448999997, 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304765301000003, 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:27.327 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:27.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:27.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304754695, 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:27.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304744357000004, 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:27.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304734350000004, 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:27.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:27.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:27.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304724398999994, 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:27.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.349 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.304709091000007, 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.350 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:27.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.302733204000006, 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.355 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:27.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:27.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.302722247999995, 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:27.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:27.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.302709758999995, 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:27.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:27.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:27.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.302699296, 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:27.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:27.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:27.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:27.373 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:27.374 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:27.375 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:27.372+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:27.475 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:27.473+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:27.488 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:27.488 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:27.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:27.576 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:27.574+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:27.645 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:27.645 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:27.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:27.697 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:27.695+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:27.760 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:27.760 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:27.769 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:27.810 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:27.808+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:27.879 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:27.877+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:27.917 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:27.918 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:27.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:27.944 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:47:27.960 INFO:teuthology.orchestra.run.smithi165.stdout:flattening image image_client.0-clone29 2024-09-17T21:47:28.002 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:28.000+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:28.058 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:28.058 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:28.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.f 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.d 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.1f 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.1e 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.16 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.13 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.12 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.11 2024-09-17T21:47:28.081 INFO:teuthology.orchestra.run.smithi053.stdout:3.15 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:2.7 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:2.5 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:3.a 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:2.6 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:2.0 2024-09-17T21:47:28.082 INFO:teuthology.orchestra.run.smithi053.stdout:1.0 2024-09-17T21:47:28.108 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:28.106+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:28.182 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:28.179+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:28.215 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:28.216 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:28.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:28.282 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:28.280+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:28.332 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:28.332 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:28.341 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:28.382 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:28.380+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:28.470 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-17T21:47:28.483 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:28.481+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:28.490 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:28.490 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:28.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:28.526 INFO:teuthology.orchestra.run.smithi165.stdout:3619 punch from 0x5219200 to 0x521c200, (0x3000 bytes) 2024-09-17T21:47:28.541 INFO:teuthology.orchestra.run.smithi165.stdout:3620 read 0x7024200 thru 0x702b803 (0x7604 bytes) 2024-09-17T21:47:28.541 INFO:teuthology.orchestra.run.smithi165.stdout:3621 write 0xe360400 thru 0xe36d71a (0xd31b bytes) 2024-09-17T21:47:28.541 INFO:teuthology.orchestra.run.smithi165.stdout:3622 write 0xcbb0400 thru 0xcbbb457 (0xb058 bytes) 2024-09-17T21:47:28.542 INFO:teuthology.orchestra.run.smithi165.stdout:3625 flatten 2024-09-17T21:47:28.577 INFO:teuthology.orchestra.run.smithi165.stdout:3626 write 0x3f08200 thru 0x3f0d943 (0x5744 bytes) 2024-09-17T21:47:28.580 INFO:teuthology.orchestra.run.smithi165.stdout:3628 read 0x967200 thru 0x96c6c0 (0x54c1 bytes) 2024-09-17T21:47:28.581 INFO:teuthology.orchestra.run.smithi165.stdout:3630 write 0xca5c00 thru 0xcb0a78 (0xae79 bytes) 2024-09-17T21:47:28.582 INFO:teuthology.orchestra.run.smithi165.stdout:3631 read 0x1639600 thru 0x163a2b3 (0xcb4 bytes) 2024-09-17T21:47:28.583 INFO:teuthology.orchestra.run.smithi165.stdout:3632 trunc from 0xe673400 to 0xe499c00 2024-09-17T21:47:28.584 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:28.581+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:28.599 INFO:teuthology.orchestra.run.smithi165.stdout:3633 write 0x98af800 thru 0x98b22d7 (0x2ad8 bytes) 2024-09-17T21:47:28.599 INFO:teuthology.orchestra.run.smithi165.stdout:3634 read 0x8498800 thru 0x84a7aa1 (0xf2a2 bytes) 2024-09-17T21:47:28.600 INFO:teuthology.orchestra.run.smithi165.stdout:3635 punch from 0x6592400 to 0x659f200, (0xce00 bytes) 2024-09-17T21:47:28.600 INFO:teuthology.orchestra.run.smithi165.stdout:3637 read 0xd104600 thru 0xd10aa4b (0x644c bytes) 2024-09-17T21:47:28.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --op export --pgid 3.a --file /var/log/ceph/exp.3.a.3 2024-09-17T21:47:28.601 INFO:teuthology.orchestra.run.smithi165.stdout:3638 clone 31 order 21 su 32768 sc 11 2024-09-17T21:47:28.659 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:28.659 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:28.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:28.685 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:28.683+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:28.786 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:28.784+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:28.791 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:28.791 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:28.791 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:28.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:28.887 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:28.885+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:28.906 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:28.907 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:28.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:28.988 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:28.986+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:29.026 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.069709372999995, 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:29.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:29.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.055723317000002, 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.031 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:29.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.053762109999994, 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:29.035 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:29.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:29.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.039 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.051728753000006, 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:29.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:29.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.049750834999998, 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.049728040999995, 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:29.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.043818978000004, 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:29.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:29.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:29.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.041787919000001, 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.057 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:29.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.041761344999998, 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.061 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:29.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:29.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.041151552000002, 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.041133966000004, 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:29.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:29.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.041119008999999, 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:29.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:29.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.325437520999998, 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.325406900000004, 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:29.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:29.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.325382282000007, 2024-09-17T21:47:29.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:29.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.325360334999999, 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:29.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:29.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.107 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 71.325338865000006, 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:29.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:29.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:29.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:29.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:29.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.283922062000002, 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:29.113 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:29.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.282877601999999, 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.118 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.0816922710000001, 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:29.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:29.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.130 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.130 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:29.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:29.132 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:29.129+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:29.232 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:29.230+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:29.282 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:29.283 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:29.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:29.333 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:29.331+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:29.378 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-17T21:47:29.402 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:29.402 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:29.402 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:29.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:29.433 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:29.431+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:29.526 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:29.526 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552380560000003, 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552358067, 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:29.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:29.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552333806999997, 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.539 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552307924000004, 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.540 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:29.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:29.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552282923000007, 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:29.544 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:29.548 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552243124, 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:29.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:29.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:29.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552227379000001, 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552215844000003, 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:29.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.552205544000003, 2024-09-17T21:47:29.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:29.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:29.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550234301000003, 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:29.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:29.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550221152999995, 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:29.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:29.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:29.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550210547000006, 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550200208999996, 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:29.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:29.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550190201999996, 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:29.582 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:29.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:29.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550180251, 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:29.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.550164942999999, 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:29.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.548189055999998, 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:29.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:29.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.548178100000001, 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:29.603 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.548165611000002, 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:29.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:29.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.548155148000006, 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:29.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:29.612 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:29.613 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:29.610+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:29.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:29.714 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:29.712+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:29.731 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:29.731 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:29.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:29.815 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:29.812+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:29.889 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:29.889 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:29.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:29.941 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:29.939+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:29.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:47:30.002 INFO:teuthology.orchestra.run.smithi165.stdout:3640 write 0x7b8a800 thru 0x7b90b08 (0x6309 bytes) 2024-09-17T21:47:30.003 INFO:teuthology.orchestra.run.smithi165.stdout:3641 read 0x3d50a00 thru 0x3d52800 (0x1e01 bytes) 2024-09-17T21:47:30.004 INFO:teuthology.orchestra.run.smithi165.stdout:3642 punch from 0xe386800 to 0xe392a00, (0xc200 bytes) 2024-09-17T21:47:30.007 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.007 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.016 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:30.014+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:30.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:30.024 INFO:teuthology.orchestra.run.smithi165.stdout:3643 read 0x1206a00 thru 0x12098a1 (0x2ea2 bytes) 2024-09-17T21:47:30.024 INFO:teuthology.orchestra.run.smithi165.stdout:3645 read 0x1b4ca00 thru 0x1b53e3d (0x743e bytes) 2024-09-17T21:47:30.024 INFO:teuthology.orchestra.run.smithi165.stdout:3647 read 0x8238a00 thru 0x8242c03 (0xa204 bytes) 2024-09-17T21:47:30.025 INFO:teuthology.orchestra.run.smithi165.stdout:3649 write 0x415000 thru 0x41ff3c (0xaf3d bytes) 2024-09-17T21:47:30.025 INFO:teuthology.orchestra.run.smithi165.stdout:3650 read 0x576de00 thru 0x57784de (0xa6df bytes) 2024-09-17T21:47:30.026 INFO:teuthology.orchestra.run.smithi165.stdout:3654 write 0x85e2800 thru 0x85e3fae (0x17af bytes) 2024-09-17T21:47:30.027 INFO:teuthology.orchestra.run.smithi165.stdout:3655 read 0x3d6d400 thru 0x3d772c0 (0x9ec1 bytes) 2024-09-17T21:47:30.028 INFO:teuthology.orchestra.run.smithi165.stdout:3658 write 0xd455e00 thru 0xd4641d9 (0xe3da bytes) 2024-09-17T21:47:30.035 INFO:teuthology.orchestra.run.smithi165.stdout:3660 trunc from 0xe499c00 to 0x7a0e200 2024-09-17T21:47:30.115 INFO:teuthology.orchestra.run.smithi165.stdout:3661 write 0xbee8000 thru 0xbef00a6 (0x80a7 bytes) 2024-09-17T21:47:30.117 INFO:teuthology.orchestra.run.smithi165.stdout:3663 trunc from 0xbef0200 to 0xb300a00 2024-09-17T21:47:30.117 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:30.115+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:30.134 INFO:teuthology.orchestra.run.smithi165.stdout:3665 read 0x1f6da00 thru 0x1f741da (0x67db bytes) 2024-09-17T21:47:30.139 INFO:teuthology.orchestra.run.smithi165.stdout:3667 trunc from 0xb300a00 to 0xe5b8200 2024-09-17T21:47:30.149 INFO:teuthology.orchestra.run.smithi165.stdout:3669 write 0x78d8200 thru 0x78e1be7 (0x99e8 bytes) 2024-09-17T21:47:30.150 INFO:teuthology.orchestra.run.smithi165.stdout:3673 read 0xb465600 thru 0xb46a633 (0x5034 bytes) 2024-09-17T21:47:30.150 INFO:teuthology.orchestra.run.smithi165.stdout:3676 read 0x795de00 thru 0x79640c9 (0x62ca bytes) 2024-09-17T21:47:30.152 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.152 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.153 INFO:teuthology.orchestra.run.smithi165.stdout:3677 read 0x697ea00 thru 0x698c01d (0xd61e bytes) 2024-09-17T21:47:30.153 INFO:teuthology.orchestra.run.smithi165.stdout:3679 read 0x73eba00 thru 0x73f87f5 (0xcdf6 bytes) 2024-09-17T21:47:30.153 INFO:teuthology.orchestra.run.smithi165.stdout:3681 read 0x950c600 thru 0x95193d5 (0xcdd6 bytes) 2024-09-17T21:47:30.153 INFO:teuthology.orchestra.run.smithi165.stdout:3684 read 0xc03d800 thru 0xc04abd6 (0xd3d7 bytes) 2024-09-17T21:47:30.154 INFO:teuthology.orchestra.run.smithi165.stdout:3685 write 0x2b97c00 thru 0x2ba46d4 (0xcad5 bytes) 2024-09-17T21:47:30.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:30.164 INFO:teuthology.orchestra.run.smithi165.stdout:3686 trunc from 0xe5b8200 to 0x5e9d600 2024-09-17T21:47:30.217 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:30.215+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:30.224 INFO:teuthology.orchestra.run.smithi165.stdout:3687 write 0x3ab7200 thru 0x3aba3e8 (0x31e9 bytes) 2024-09-17T21:47:30.226 INFO:teuthology.orchestra.run.smithi165.stdout:3690 read 0x5d3b400 thru 0x5d3fcf0 (0x48f1 bytes) 2024-09-17T21:47:30.226 INFO:teuthology.orchestra.run.smithi165.stdout:3693 write 0x4936600 thru 0x4942f14 (0xc915 bytes) 2024-09-17T21:47:30.229 INFO:teuthology.orchestra.run.smithi165.stdout:3694 read 0x3006800 thru 0x300e6b5 (0x7eb6 bytes) 2024-09-17T21:47:30.231 INFO:teuthology.orchestra.run.smithi165.stdout:3696 trunc from 0x5e9d600 to 0x292e600 2024-09-17T21:47:30.303 INFO:teuthology.orchestra.run.smithi053.stderr:Exporting 3.a info 3.a( v 95'186 (0'0,95'186] local-lis/les=80/81 n=22 ec=21/21 lis/c=80/80 les/c/f=81/81/0 sis=80) 2024-09-17T21:47:30.308 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.309 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.311 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:50216c96:::rbd_data.1112184ea9c8.0000000000000077:head# 2024-09-17T21:47:30.311 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:5084e2dd:::rbd_data.1112d625dbc0.0000000000000042:1e# 2024-09-17T21:47:30.312 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:5084e2dd:::rbd_data.1112d625dbc0.0000000000000042:head# 2024-09-17T21:47:30.312 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:50a54057:::rbd_data.11121f8e87bb.0000000000000125:head# 2024-09-17T21:47:30.312 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:510afc60:::rbd_data.11121f8e87bb.0000000000000008:head# 2024-09-17T21:47:30.313 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:52c4a3ad:::rbd_data.1112bc1b2568.0000000000000004:head# 2024-09-17T21:47:30.313 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:52c7d6d0:::rbd_data.11128c113d55.0000000000000011:head# 2024-09-17T21:47:30.313 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:52f2ba4c:::rbd_data.1112a93a0526.0000000000000002:head# 2024-09-17T21:47:30.313 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:535385f0:::rbd_children:head# 2024-09-17T21:47:30.313 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:539e9b35:::rbd_data.11128c113d55.0000000000000004:head# 2024-09-17T21:47:30.315 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:545786ae:::rbd_id.image_client.0-clone13:head# 2024-09-17T21:47:30.315 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:54f8743c:::rbd_data.1112fae1757a.0000000000000130:1f# 2024-09-17T21:47:30.315 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:54f8743c:::rbd_data.1112fae1757a.0000000000000130:head# 2024-09-17T21:47:30.315 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:55056370:::rbd_data.11124390a7c3.000000000000000d:head# 2024-09-17T21:47:30.315 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:556d2800:::rbd_data.11121f8e87bb.000000000000003c:head# 2024-09-17T21:47:30.316 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:559e4506:::rbd_data.11121f8e87bb.0000000000000123:head# 2024-09-17T21:47:30.316 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:56770158:::rbd_data.11121f8e87bb.0000000000000091:head# 2024-09-17T21:47:30.316 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:568819d6:::rbd_data.1112e01b91a8.0000000000000017:9# 2024-09-17T21:47:30.317 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:568819d6:::rbd_data.1112e01b91a8.0000000000000017:head# 2024-09-17T21:47:30.317 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:56921cc9:::rbd_data.11121f8e87bb.000000000000005f:head# 2024-09-17T21:47:30.319 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:569eea4d:::rbd_data.1112184ea9c8.0000000000000076:head# 2024-09-17T21:47:30.319 INFO:teuthology.orchestra.run.smithi053.stderr:Read #3:57403a42:::rbd_object_map.1112a8bc19be.000000000000000b:head# 2024-09-17T21:47:30.319 INFO:teuthology.orchestra.run.smithi053.stderr:Export successful 2024-09-17T21:47:30.319 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:30.319 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:30.316+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:30.327 INFO:teuthology.orchestra.run.smithi165.stdout:3697 write 0x6bf9000 thru 0x6c03120 (0xa121 bytes) 2024-09-17T21:47:30.328 INFO:teuthology.orchestra.run.smithi165.stdout:3698 read 0x1ba1c00 thru 0x1bad342 (0xb743 bytes) 2024-09-17T21:47:30.328 INFO:teuthology.orchestra.run.smithi165.stdout:3699 punch from 0xd7ce00 to 0xd89000, (0xc200 bytes) 2024-09-17T21:47:30.358 INFO:teuthology.orchestra.run.smithi165.stdout:3704 write 0x9dd4200 thru 0x9ddeea0 (0xaca1 bytes) 2024-09-17T21:47:30.360 INFO:teuthology.orchestra.run.smithi165.stdout:3705 write 0x184da00 thru 0x184e66c (0xc6d bytes) 2024-09-17T21:47:30.360 INFO:teuthology.orchestra.run.smithi165.stdout:3706 read 0x9ae8600 thru 0x9aedcba (0x56bb bytes) 2024-09-17T21:47:30.363 INFO:teuthology.orchestra.run.smithi165.stdout:3707 read 0x3b4a200 thru 0x3b5a0ac (0xfead bytes) 2024-09-17T21:47:30.363 INFO:teuthology.orchestra.run.smithi165.stdout:3711 punch from 0x279d000 to 0x27a7600, (0xa600 bytes) 2024-09-17T21:47:30.363 INFO:teuthology.orchestra.run.smithi165.stdout:3713 read 0x204f600 thru 0x2054aa2 (0x54a3 bytes) 2024-09-17T21:47:30.363 INFO:teuthology.orchestra.run.smithi165.stdout:3715 punch from 0x42d3e00 to 0x42dcc00, (0x8e00 bytes) 2024-09-17T21:47:30.373 INFO:teuthology.orchestra.run.smithi165.stdout:3717 trunc from 0x9ddf000 to 0x38bc00 2024-09-17T21:47:30.419 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:30.416+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:30.433 INFO:teuthology.orchestra.run.smithi165.stdout:3718 write 0x8f43800 thru 0x8f51eef (0xe6f0 bytes) 2024-09-17T21:47:30.437 INFO:teuthology.orchestra.run.smithi165.stdout:3721 write 0x73ac600 thru 0x73b725c (0xac5d bytes) 2024-09-17T21:47:30.452 INFO:teuthology.orchestra.run.smithi165.stdout:3722 write 0xc6aaa00 thru 0xc6b1db6 (0x73b7 bytes) 2024-09-17T21:47:30.453 INFO:teuthology.orchestra.run.smithi165.stdout:3724 read 0x2461600 thru 0x2464dbb (0x37bc bytes) 2024-09-17T21:47:30.453 INFO:teuthology.orchestra.run.smithi165.stdout:3725 write 0x6646800 thru 0x664a438 (0x3c39 bytes) 2024-09-17T21:47:30.453 INFO:teuthology.orchestra.run.smithi165.stdout:3727 punch from 0x995e800 to 0x9960400, (0x1c00 bytes) 2024-09-17T21:47:30.454 INFO:teuthology.orchestra.run.smithi165.stdout:3729 read 0x7999600 thru 0x79a8dff (0xf800 bytes) 2024-09-17T21:47:30.454 INFO:teuthology.orchestra.run.smithi165.stdout:3730 punch from 0xab88a00 to 0xab98800, (0xfe00 bytes) 2024-09-17T21:47:30.455 INFO:teuthology.orchestra.run.smithi165.stdout:3731 write 0x3a5bc00 thru 0x3a63654 (0x7a55 bytes) 2024-09-17T21:47:30.464 INFO:teuthology.orchestra.run.smithi165.stdout:3732 trunc from 0xc6b1e00 to 0x463a200 2024-09-17T21:47:30.488 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.488 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:30.502 INFO:teuthology.orchestra.run.smithi165.stdout:3733 write 0x5974200 thru 0x597c9dd (0x87de bytes) 2024-09-17T21:47:30.508 INFO:teuthology.orchestra.run.smithi165.stdout:3735 trunc from 0x597ca00 to 0xb8aa00 2024-09-17T21:47:30.520 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:30.517+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:30.538 INFO:teuthology.orchestra.run.smithi165.stdout:3739 trunc from 0xb8aa00 to 0x4a0c600 2024-09-17T21:47:30.554 INFO:teuthology.orchestra.run.smithi165.stdout:3743 read 0x2003600 thru 0x200a415 (0x6e16 bytes) 2024-09-17T21:47:30.572 INFO:teuthology.orchestra.run.smithi165.stdout:3747 write 0xd330600 thru 0xd33a9c1 (0xa3c2 bytes) 2024-09-17T21:47:30.578 INFO:teuthology.orchestra.run.smithi165.stdout:3749 trunc from 0xd33aa00 to 0x9828400 2024-09-17T21:47:30.594 INFO:teuthology.orchestra.run.smithi165.stdout:3750 read 0x70f9400 thru 0x710746f (0xe070 bytes) 2024-09-17T21:47:30.604 INFO:teuthology.orchestra.run.smithi165.stdout:3751 write 0x9bd9400 thru 0x9be5dfa (0xc9fb bytes) 2024-09-17T21:47:30.611 INFO:teuthology.orchestra.run.smithi165.stdout:3758 read 0x12ab800 thru 0x12b7f5a (0xc75b bytes) 2024-09-17T21:47:30.612 INFO:teuthology.orchestra.run.smithi165.stdout:3760 read 0x8caaa00 thru 0x8cac474 (0x1a75 bytes) 2024-09-17T21:47:30.616 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.616 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.620 INFO:teuthology.orchestra.run.smithi165.stdout:3761 trunc from 0x9be5e00 to 0x32e2200 2024-09-17T21:47:30.621 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:30.618+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:30.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:30.650 INFO:teuthology.orchestra.run.smithi165.stdout:3763 write 0x8db1600 thru 0x8db3c71 (0x2672 bytes) 2024-09-17T21:47:30.672 INFO:teuthology.orchestra.run.smithi165.stdout:3764 write 0xcef8c00 thru 0xcefa480 (0x1881 bytes) 2024-09-17T21:47:30.672 INFO:teuthology.orchestra.run.smithi165.stdout:3765 punch from 0xb58ec00 to 0xb598a00, (0x9e00 bytes) 2024-09-17T21:47:30.672 INFO:teuthology.orchestra.run.smithi165.stdout:3768 write 0x8c10c00 thru 0x8c1dfc2 (0xd3c3 bytes) 2024-09-17T21:47:30.693 INFO:teuthology.orchestra.run.smithi165.stdout:3769 write 0xe6c4c00 thru 0xe6ca537 (0x5938 bytes) 2024-09-17T21:47:30.693 INFO:teuthology.orchestra.run.smithi165.stdout:3770 read 0xbc2e600 thru 0xbc38bda (0xa5db bytes) 2024-09-17T21:47:30.693 INFO:teuthology.orchestra.run.smithi165.stdout:3771 write 0x7652600 thru 0x765ddae (0xb7af bytes) 2024-09-17T21:47:30.693 INFO:teuthology.orchestra.run.smithi165.stdout:3772 punch from 0xa383200 to 0xa38ae00, (0x7c00 bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3777 read 0x696f200 thru 0x69753bc (0x61bd bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3779 write 0x2e36a00 thru 0x2e430b5 (0xc6b6 bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3780 read 0xcf28a00 thru 0xcf36a82 (0xe083 bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3781 write 0x7d81800 thru 0x7d88791 (0x6f92 bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3783 punch from 0x3759c00 to 0x3766e00, (0xd200 bytes) 2024-09-17T21:47:30.694 INFO:teuthology.orchestra.run.smithi165.stdout:3784 read 0x6f0c000 thru 0x6f11c95 (0x5c96 bytes) 2024-09-17T21:47:30.695 INFO:teuthology.orchestra.run.smithi165.stdout:3788 read 0x7498200 thru 0x749ada6 (0x2ba7 bytes) 2024-09-17T21:47:30.696 INFO:teuthology.orchestra.run.smithi165.stdout:3789 trunc from 0xe6ca600 to 0xc7fd600 2024-09-17T21:47:30.719 INFO:teuthology.orchestra.run.smithi165.stdout:3792 write 0x6eae400 thru 0x6eb9689 (0xb28a bytes) 2024-09-17T21:47:30.720 INFO:teuthology.orchestra.run.smithi165.stdout:3793 punch from 0x9872200 to 0x987aa00, (0x8800 bytes) 2024-09-17T21:47:30.721 INFO:teuthology.orchestra.run.smithi165.stdout:3794 write 0x8139400 thru 0x8143f84 (0xab85 bytes) 2024-09-17T21:47:30.721 INFO:teuthology.orchestra.run.smithi165.stdout:3796 read 0x23c7400 thru 0x23d1464 (0xa065 bytes) 2024-09-17T21:47:30.721 INFO:teuthology.orchestra.run.smithi165.stdout:3797 read 0x712f400 thru 0x7131bce (0x27cf bytes) 2024-09-17T21:47:30.722 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:30.719+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:30.722 INFO:teuthology.orchestra.run.smithi165.stdout:3799 punch from 0x5186a00 to 0x5191200, (0xa800 bytes) 2024-09-17T21:47:30.722 INFO:teuthology.orchestra.run.smithi165.stdout:3802 write 0x4870200 thru 0x487d590 (0xd391 bytes) 2024-09-17T21:47:30.722 INFO:teuthology.orchestra.run.smithi165.stdout:3803 punch from 0xf25600 to 0xf2a800, (0x5200 bytes) 2024-09-17T21:47:30.737 INFO:teuthology.orchestra.run.smithi165.stdout:3804 write 0xd963e00 thru 0xd96cd19 (0x8f1a bytes) 2024-09-17T21:47:30.739 INFO:teuthology.orchestra.run.smithi165.stdout:3806 trunc from 0xd96ce00 to 0xc464a00 2024-09-17T21:47:30.756 INFO:teuthology.orchestra.run.smithi165.stdout:3807 trunc from 0xc464a00 to 0xb8adc00 2024-09-17T21:47:30.768 INFO:teuthology.orchestra.run.smithi165.stdout:3808 read 0x1a26600 thru 0x1a298cf (0x32d0 bytes) 2024-09-17T21:47:30.769 INFO:teuthology.orchestra.run.smithi165.stdout:3809 read 0x57ede00 thru 0x57f6677 (0x8878 bytes) 2024-09-17T21:47:30.769 INFO:teuthology.orchestra.run.smithi165.stdout:3814 read 0xabb8a00 thru 0xabbb295 (0x2896 bytes) 2024-09-17T21:47:30.769 INFO:teuthology.orchestra.run.smithi165.stdout:3815 read 0x5740400 thru 0x574f2fd (0xeefe bytes) 2024-09-17T21:47:30.770 INFO:teuthology.orchestra.run.smithi165.stdout:3816 read 0x9555800 thru 0x9564c6a (0xf46b bytes) 2024-09-17T21:47:30.770 INFO:teuthology.orchestra.run.smithi165.stdout:3817 punch from 0x5fc3200 to 0x5fd1600, (0xe400 bytes) 2024-09-17T21:47:30.770 INFO:teuthology.orchestra.run.smithi165.stdout:3818 read 0x5b23200 thru 0x5b32c7f (0xfa80 bytes) 2024-09-17T21:47:30.771 INFO:teuthology.orchestra.run.smithi165.stdout:3819 punch from 0x829fe00 to 0x82a9200, (0x9400 bytes) 2024-09-17T21:47:30.771 INFO:teuthology.orchestra.run.smithi165.stdout:3820 write 0x1c43600 thru 0x1c51bb8 (0xe5b9 bytes) 2024-09-17T21:47:30.771 INFO:teuthology.orchestra.run.smithi165.stdout:3824 read 0x17ce000 thru 0x17d7c2d (0x9c2e bytes) 2024-09-17T21:47:30.774 INFO:teuthology.orchestra.run.smithi165.stdout:3826 trunc from 0xb8adc00 to 0x8bf8600 2024-09-17T21:47:30.791 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.791 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:30.808 INFO:teuthology.orchestra.run.smithi165.stdout:3832 write 0xed7c600 thru 0xed866ba (0xa0bb bytes) 2024-09-17T21:47:30.810 INFO:teuthology.orchestra.run.smithi165.stdout:3833 read 0xb045a00 thru 0xb049e19 (0x441a bytes) 2024-09-17T21:47:30.810 INFO:teuthology.orchestra.run.smithi165.stdout:3834 punch from 0xeb0a800 to 0xeb0c800, (0x2000 bytes) 2024-09-17T21:47:30.816 INFO:teuthology.orchestra.run.smithi165.stdout:3836 trunc from 0xed86800 to 0x9ffc200 2024-09-17T21:47:30.821 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:30.819+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:30.835 INFO:teuthology.orchestra.run.smithi165.stdout:3838 write 0x2102e00 thru 0x21082fc (0x54fd bytes) 2024-09-17T21:47:30.841 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal --force --op remove --pgid 3.a 2024-09-17T21:47:30.857 INFO:teuthology.orchestra.run.smithi165.stdout:3839 write 0xe13c400 thru 0xe142fb9 (0x6bba bytes) 2024-09-17T21:47:30.857 INFO:teuthology.orchestra.run.smithi165.stdout:3842 write 0xa63a400 thru 0xa63c677 (0x2278 bytes) 2024-09-17T21:47:30.857 INFO:teuthology.orchestra.run.smithi165.stdout:3843 punch from 0x7cd5800 to 0x7ce0e00, (0xb600 bytes) 2024-09-17T21:47:30.869 INFO:teuthology.orchestra.run.smithi165.stdout:3850 trunc from 0xe143000 to 0x3230600 2024-09-17T21:47:30.900 INFO:teuthology.orchestra.run.smithi165.stdout:3852 trunc from 0x3230600 to 0x88f8e00 2024-09-17T21:47:30.909 INFO:teuthology.orchestra.run.smithi165.stdout:3853 write 0x6d4d000 thru 0x6d518d1 (0x48d2 bytes) 2024-09-17T21:47:30.909 INFO:teuthology.orchestra.run.smithi165.stdout:3855 write 0x1cf600 thru 0x1de014 (0xea15 bytes) 2024-09-17T21:47:30.910 INFO:teuthology.orchestra.run.smithi165.stdout:3856 read 0x39be200 thru 0x39c6a58 (0x8859 bytes) 2024-09-17T21:47:30.910 INFO:teuthology.orchestra.run.smithi165.stdout:3857 punch from 0xe72c00 to 0xe77000, (0x4400 bytes) 2024-09-17T21:47:30.911 INFO:teuthology.orchestra.run.smithi165.stdout:3860 write 0x1807e00 thru 0x180d1f6 (0x53f7 bytes) 2024-09-17T21:47:30.911 INFO:teuthology.orchestra.run.smithi165.stdout:3865 punch from 0x137a400 to 0x1386000, (0xbc00 bytes) 2024-09-17T21:47:30.911 INFO:teuthology.orchestra.run.smithi165.stdout:3866 read 0x7e00c00 thru 0x7e028fe (0x1cff bytes) 2024-09-17T21:47:30.914 INFO:teuthology.orchestra.run.smithi165.stdout:3867 trunc from 0x88f8e00 to 0xb0f1200 2024-09-17T21:47:30.922 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:30.920+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:30.924 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:30.924 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:30.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:30.934 INFO:teuthology.orchestra.run.smithi165.stdout:3868 trunc from 0xb0f1200 to 0x4b9c600 2024-09-17T21:47:30.947 INFO:teuthology.orchestra.run.smithi165.stdout:3869 punch from 0x4290400 to 0x4292200, (0x1e00 bytes) 2024-09-17T21:47:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:47:30.970 INFO:teuthology.orchestra.run.smithi165.stdout:3871 write 0xcd76a00 thru 0xcd793b7 (0x29b8 bytes) 2024-09-17T21:47:30.977 INFO:teuthology.orchestra.run.smithi165.stdout:3872 write 0x89a4e00 thru 0x89b048d (0xb68e bytes) 2024-09-17T21:47:30.977 INFO:teuthology.orchestra.run.smithi165.stdout:3876 write 0xcbe2400 thru 0xcbe3a94 (0x1695 bytes) 2024-09-17T21:47:30.978 INFO:teuthology.orchestra.run.smithi165.stdout:3877 read 0x1acae00 thru 0x1ad881a (0xda1b bytes) 2024-09-17T21:47:30.978 INFO:teuthology.orchestra.run.smithi165.stdout:3878 read 0x1d9a200 thru 0x1da8f0e (0xed0f bytes) 2024-09-17T21:47:30.979 INFO:teuthology.orchestra.run.smithi165.stdout:3879 punch from 0x6003c00 to 0x6010000, (0xc400 bytes) 2024-09-17T21:47:30.997 INFO:teuthology.orchestra.run.smithi165.stdout:3880 write 0xdc3be00 thru 0xdc440d5 (0x82d6 bytes) 2024-09-17T21:47:31.003 INFO:teuthology.orchestra.run.smithi165.stdout:3881 write 0x7fccc00 thru 0x7fd5163 (0x8564 bytes) 2024-09-17T21:47:31.003 INFO:teuthology.orchestra.run.smithi165.stdout:3884 punch from 0xd99e200 to 0xd9ab800, (0xd600 bytes) 2024-09-17T21:47:31.004 INFO:teuthology.orchestra.run.smithi165.stdout:3886 read 0x8966800 thru 0x89677ab (0xfac bytes) 2024-09-17T21:47:31.004 INFO:teuthology.orchestra.run.smithi165.stdout:3889 write 0x68ba00 thru 0x695fdc (0xa5dd bytes) 2024-09-17T21:47:31.006 INFO:teuthology.orchestra.run.smithi165.stdout:3891 write 0xbfa9000 thru 0xbfac8dc (0x38dd bytes) 2024-09-17T21:47:31.006 INFO:teuthology.orchestra.run.smithi165.stdout:3892 write 0x8f86400 thru 0x8f9111c (0xad1d bytes) 2024-09-17T21:47:31.006 INFO:teuthology.orchestra.run.smithi165.stdout:3893 write 0x1943e00 thru 0x194d9d7 (0x9bd8 bytes) 2024-09-17T21:47:31.006 INFO:teuthology.orchestra.run.smithi165.stdout:3894 write 0xd6a3600 thru 0xd6b2152 (0xeb53 bytes) 2024-09-17T21:47:31.006 INFO:teuthology.orchestra.run.smithi165.stdout:3895 read 0xbc55400 thru 0xbc5afb2 (0x5bb3 bytes) 2024-09-17T21:47:31.008 INFO:teuthology.orchestra.run.smithi165.stdout:3898 trunc from 0xdc44200 to 0xbf59600 2024-09-17T21:47:31.023 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:31.021+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:31.031 INFO:teuthology.orchestra.run.smithi165.stdout:3899 read 0x33dc600 thru 0x33e4548 (0x7f49 bytes) 2024-09-17T21:47:31.039 INFO:teuthology.orchestra.run.smithi165.stdout:3902 trunc from 0xbf59600 to 0x6766e00 2024-09-17T21:47:31.053 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:31.053 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:31.053 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:31.053 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:31.054 INFO:teuthology.orchestra.run.smithi165.stdout:3905 punch from 0x1da2e00 to 0x1da4200, (0x1400 bytes) 2024-09-17T21:47:31.060 INFO:teuthology.orchestra.run.smithi165.stdout:3915 read 0xdc8a00 thru 0xdd42a0 (0xb8a1 bytes) 2024-09-17T21:47:31.060 INFO:teuthology.orchestra.run.smithi165.stdout:3917 read 0x346b400 thru 0x34755a2 (0xa1a3 bytes) 2024-09-17T21:47:31.061 INFO:teuthology.orchestra.run.smithi165.stdout:3923 read 0x18c4400 thru 0x18d4360 (0xff61 bytes) 2024-09-17T21:47:31.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:31.063 INFO:teuthology.orchestra.run.smithi165.stdout:3925 read 0x2638c00 thru 0x264160e (0x8a0f bytes) 2024-09-17T21:47:31.063 INFO:teuthology.orchestra.run.smithi165.stdout:3927 read 0xc33200 thru 0xc3cd0e (0x9b0f bytes) 2024-09-17T21:47:31.063 INFO:teuthology.orchestra.run.smithi165.stdout:3928 read 0x1845800 thru 0x18484a1 (0x2ca2 bytes) 2024-09-17T21:47:31.078 INFO:teuthology.orchestra.run.smithi165.stdout:3929 write 0xbc96600 thru 0xbc9cc1d (0x661e bytes) 2024-09-17T21:47:31.086 INFO:teuthology.orchestra.run.smithi165.stdout:3932 read 0x4c29200 thru 0x4c2f6eb (0x64ec bytes) 2024-09-17T21:47:31.086 INFO:teuthology.orchestra.run.smithi165.stdout:3936 read 0x4923600 thru 0x4928230 (0x4c31 bytes) 2024-09-17T21:47:31.086 INFO:teuthology.orchestra.run.smithi165.stdout:3938 write 0x718a800 thru 0x71923aa (0x7bab bytes) 2024-09-17T21:47:31.086 INFO:teuthology.orchestra.run.smithi165.stdout:3943 write 0x7d5b200 thru 0x7d5ee85 (0x3c86 bytes) 2024-09-17T21:47:31.086 INFO:teuthology.orchestra.run.smithi165.stdout:3945 read 0xacd600 thru 0xad73a5 (0x9da6 bytes) 2024-09-17T21:47:31.087 INFO:teuthology.orchestra.run.smithi165.stdout:3947 write 0x3691e00 thru 0x3693630 (0x1831 bytes) 2024-09-17T21:47:31.087 INFO:teuthology.orchestra.run.smithi165.stdout:3949 punch from 0x46a9a00 to 0x46b9000, (0xf600 bytes) 2024-09-17T21:47:31.087 INFO:teuthology.orchestra.run.smithi165.stdout:3950 read 0x480a200 thru 0x4817ef0 (0xdcf1 bytes) 2024-09-17T21:47:31.089 INFO:teuthology.orchestra.run.smithi165.stdout:3951 write 0xaef6a00 thru 0xaf0159f (0xaba0 bytes) 2024-09-17T21:47:31.089 INFO:teuthology.orchestra.run.smithi165.stdout:3952 punch from 0x6cd0e00 to 0x6cdb600, (0xa800 bytes) 2024-09-17T21:47:31.109 INFO:teuthology.orchestra.run.smithi165.stdout:3954 write 0xca1de00 thru 0xca24e24 (0x7025 bytes) 2024-09-17T21:47:31.115 INFO:teuthology.orchestra.run.smithi165.stdout:3955 read 0x61b400 thru 0x62962b (0xe22c bytes) 2024-09-17T21:47:31.116 INFO:teuthology.orchestra.run.smithi165.stdout:3957 trunc from 0xca25000 to 0xc24a000 2024-09-17T21:47:31.132 INFO:teuthology.orchestra.run.smithi165.stdout:3958 write 0x8e7ca00 thru 0x8e814e7 (0x4ae8 bytes) 2024-09-17T21:47:31.139 INFO:teuthology.orchestra.run.smithi165.stdout:3959 write 0x3c2de00 thru 0x3c368e3 (0x8ae4 bytes) 2024-09-17T21:47:31.141 INFO:teuthology.orchestra.run.smithi165.stdout:3960 trunc from 0xc24a000 to 0xcbeb200 2024-09-17T21:47:31.147 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:31.145+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:31.153 INFO:teuthology.orchestra.run.smithi165.stdout:3962 write 0xc30d600 thru 0xc3148a7 (0x72a8 bytes) 2024-09-17T21:47:31.160 INFO:teuthology.orchestra.run.smithi165.stdout:3963 read 0x2eb6200 thru 0x2eb65f4 (0x3f5 bytes) 2024-09-17T21:47:31.161 INFO:teuthology.orchestra.run.smithi165.stdout:3964 read 0x39a4e00 thru 0x39ac4f1 (0x76f2 bytes) 2024-09-17T21:47:31.161 INFO:teuthology.orchestra.run.smithi165.stdout:3966 punch from 0xb2ea000 to 0xb2ec200, (0x2200 bytes) 2024-09-17T21:47:31.161 INFO:teuthology.orchestra.run.smithi165.stdout:3968 read 0x7e2e000 thru 0x7e31f1c (0x3f1d bytes) 2024-09-17T21:47:31.165 INFO:teuthology.orchestra.run.smithi165.stdout:3970 trunc from 0xcbeb200 to 0x90c6e00 2024-09-17T21:47:31.179 INFO:teuthology.orchestra.run.smithi165.stdout:3973 read 0x642c600 thru 0x642cefb (0x8fc bytes) 2024-09-17T21:47:31.181 INFO:teuthology.orchestra.run.smithi165.stdout:3974 read 0x5a66600 thru 0x5a694d2 (0x2ed3 bytes) 2024-09-17T21:47:31.188 INFO:teuthology.orchestra.run.smithi165.stdout:3978 trunc from 0x90c6e00 to 0x3307600 2024-09-17T21:47:31.205 INFO:teuthology.orchestra.run.smithi165.stdout:3983 read 0xdb4200 thru 0xdb8a46 (0x4847 bytes) 2024-09-17T21:47:31.206 INFO:teuthology.orchestra.run.smithi165.stdout:3987 punch from 0x20afe00 to 0x20b5000, (0x5200 bytes) 2024-09-17T21:47:31.207 INFO:teuthology.orchestra.run.smithi165.stdout:3988 punch from 0x18d9000 to 0x18e0400, (0x7400 bytes) 2024-09-17T21:47:31.207 INFO:teuthology.orchestra.run.smithi165.stdout:3994 punch from 0x15de200 to 0x15ea600, (0xc400 bytes) 2024-09-17T21:47:31.207 INFO:teuthology.orchestra.run.smithi165.stdout:3996 read 0xf3c400 thru 0xf481d4 (0xbdd5 bytes) 2024-09-17T21:47:31.208 INFO:teuthology.orchestra.run.smithi165.stdout:3997 punch from 0xb57800 to 0xb61c00, (0xa400 bytes) 2024-09-17T21:47:31.218 INFO:teuthology.orchestra.run.smithi165.stdout:4000 write 0x4387a00 thru 0x43953b9 (0xd9ba bytes) 2024-09-17T21:47:31.225 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:31.223+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:31.225 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:31.225 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:31.226 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:31.226 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:31.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:31.239 INFO:teuthology.orchestra.run.smithi165.stdout:4001 write 0xaebdc00 thru 0xaec987f (0xbc80 bytes) 2024-09-17T21:47:31.243 INFO:teuthology.orchestra.run.smithi165.stdout:4003 trunc from 0xaec9a00 to 0xd61da00 2024-09-17T21:47:31.254 INFO:teuthology.orchestra.run.smithi165.stdout:4004 read 0x9300e00 thru 0x9305443 (0x4644 bytes) 2024-09-17T21:47:31.270 INFO:teuthology.orchestra.run.smithi165.stdout:4005 trunc from 0xd61da00 to 0x4d74e00 2024-09-17T21:47:31.292 INFO:teuthology.orchestra.run.smithi165.stdout:4007 trunc from 0x4d74e00 to 0xc337000 2024-09-17T21:47:31.313 INFO:teuthology.orchestra.run.smithi165.stdout:4008 trunc from 0xc337000 to 0x258ea00 2024-09-17T21:47:31.314 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:31.325 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:31.323+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:31.329 INFO:teuthology.orchestra.run.smithi165.stdout:4010 read 0x19ef200 thru 0x19f1fcf (0x2dd0 bytes) 2024-09-17T21:47:31.330 INFO:teuthology.orchestra.run.smithi165.stdout:4011 clone 32 order 21 su 65536 sc 5 2024-09-17T21:47:31.336 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:31.336 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:31.353 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:31.353 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:31.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:31.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:31.386 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:31.386 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:31.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:31.427 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:31.425+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:31.503 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:31.504 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:31.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:31.557 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:31.555+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:31.619 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:31.619 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:31.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:31.629 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:31.627+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:31.730 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:31.728+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:31.796 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:31.796 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:31.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:31.853 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:31.851+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:31.925 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:31.925 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:31.932 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:31.930+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:31.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:32.033 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:32.030+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:32.082 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.082 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:32.134 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:32.132+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:32.199 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.199 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.208 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:32.225 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone31 from 0x258ea00 (overlap 0x38bc00) to 0x5ac00 2024-09-17T21:47:32.234 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:32.231+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:32.335 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:32.333+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:32.358 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.358 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:32.437 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:32.434+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:32.479 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.479 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:32.513 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-17T21:47:32.538 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:32.536+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:32.598 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.598 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:32.648 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:32.646+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:32.739 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:32.737+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:32.762 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.762 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:32.807 INFO:teuthology.orchestra.run.smithi053.stdout: marking collection for removal 2024-09-17T21:47:32.807 INFO:teuthology.orchestra.run.smithi053.stdout:setting '_remove' omap key 2024-09-17T21:47:32.807 INFO:teuthology.orchestra.run.smithi053.stdout:finish_remove_pgs 3.a_head removing 3.a 2024-09-17T21:47:32.810 INFO:teuthology.orchestra.run.smithi053.stdout:Remove successful 2024-09-17T21:47:32.840 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:32.838+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:32.889 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:32.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:32.941 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:32.938+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:32.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:47:33.041 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:33.039+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:33.047 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:33.047 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:33.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:33.142 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:33.139+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:33.149 INFO:teuthology.orchestra.run.smithi165.stdout:4015 read 0x1d8b000 thru 0x1d8fd0c (0x4d0d bytes) 2024-09-17T21:47:33.150 INFO:teuthology.orchestra.run.smithi165.stdout:4016 read 0x19a2400 thru 0x19ac843 (0xa444 bytes) 2024-09-17T21:47:33.151 INFO:teuthology.orchestra.run.smithi165.stdout:4017 read 0x590a00 thru 0x599669 (0x8c6a bytes) 2024-09-17T21:47:33.165 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:33.165 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:33.171 INFO:teuthology.orchestra.run.smithi165.stdout:4020 write 0x8202400 thru 0x8211a9d (0xf69e bytes) 2024-09-17T21:47:33.175 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:33.189 INFO:teuthology.orchestra.run.smithi165.stdout:4021 write 0xb1fb000 thru 0xb200cfb (0x5cfc bytes) 2024-09-17T21:47:33.191 INFO:teuthology.orchestra.run.smithi165.stdout:4022 write 0x1ed1200 thru 0x1ed33b2 (0x21b3 bytes) 2024-09-17T21:47:33.191 INFO:teuthology.orchestra.run.smithi165.stdout:4023 read 0x75d2c00 thru 0x75e2b98 (0xff99 bytes) 2024-09-17T21:47:33.191 INFO:teuthology.orchestra.run.smithi165.stdout:4027 write 0x1235000 thru 0x123842e (0x342f bytes) 2024-09-17T21:47:33.192 INFO:teuthology.orchestra.run.smithi165.stdout:4028 write 0x1352200 thru 0x1361336 (0xf137 bytes) 2024-09-17T21:47:33.192 INFO:teuthology.orchestra.run.smithi165.stdout:4030 read 0x9dc0c00 thru 0x9dcdfdd (0xd3de bytes) 2024-09-17T21:47:33.221 INFO:teuthology.orchestra.run.smithi165.stdout:4031 write 0xe348000 thru 0xe34ec6f (0x6c70 bytes) 2024-09-17T21:47:33.221 INFO:teuthology.orchestra.run.smithi165.stdout:4034 write 0x9d79000 thru 0x9d79bc2 (0xbc3 bytes) 2024-09-17T21:47:33.221 INFO:teuthology.orchestra.run.smithi165.stdout:4038 punch from 0xe143600 to 0xe14c000, (0x8a00 bytes) 2024-09-17T21:47:33.223 INFO:teuthology.orchestra.run.smithi165.stdout:4039 read 0xd4bfa00 thru 0xd4cb4e8 (0xbae9 bytes) 2024-09-17T21:47:33.223 INFO:teuthology.orchestra.run.smithi165.stdout:4042 write 0x6eada00 thru 0x6ebd984 (0xff85 bytes) 2024-09-17T21:47:33.223 INFO:teuthology.orchestra.run.smithi165.stdout:4045 write 0x7f3e200 thru 0x7f49ce1 (0xbae2 bytes) 2024-09-17T21:47:33.223 INFO:teuthology.orchestra.run.smithi165.stdout:4050 write 0xaf60600 thru 0xaf67dde (0x77df bytes) 2024-09-17T21:47:33.224 INFO:teuthology.orchestra.run.smithi165.stdout:4051 read 0x1a92a00 thru 0x1a9fc7f (0xd280 bytes) 2024-09-17T21:47:33.224 INFO:teuthology.orchestra.run.smithi165.stdout:4053 punch from 0x473c800 to 0x4742600, (0x5e00 bytes) 2024-09-17T21:47:33.225 INFO:teuthology.orchestra.run.smithi165.stdout:4054 write 0x4014c00 thru 0x4019cba (0x50bb bytes) 2024-09-17T21:47:33.225 INFO:teuthology.orchestra.run.smithi165.stdout:4056 read 0x1493600 thru 0x149d8c4 (0xa2c5 bytes) 2024-09-17T21:47:33.225 INFO:teuthology.orchestra.run.smithi165.stdout:4057 clone 33 order 20 su 262144 sc 2 2024-09-17T21:47:33.242 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:33.240+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:33.323 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-3 --journal-path /var/lib/ceph/osd/ceph-3/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.a.3 2024-09-17T21:47:33.327 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:33.327 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:33.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:33.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:33.377 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:33.375+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:33.444 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:33.442+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:33.481 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:33.481 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:33.483 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:33.483 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:33.492 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:33.498 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:33.498 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:33.514 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:33.515 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:33.515 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:33.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:33.532 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:33.544 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:33.542+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:33.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:33.549 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:33.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:33.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:33.640 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:33.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:33.640 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:33.640 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:33.645 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:33.643+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:33.649 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:33.746 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:33.744+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:33.755 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:33.755 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:33.764 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:33.847 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:33.845+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:33.871 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:33.871 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897656080000004, 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:33.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897633587000001, 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:33.876 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:33.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:33.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897609326999998, 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:33.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897583444000006, 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:33.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:33.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897558442999994, 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:33.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:33.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897518644000002, 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:33.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:33.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897502899000003, 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:33.900 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:33.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:33.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897491364000004, 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:33.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:33.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.897481064000004, 2024-09-17T21:47:33.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:33.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895509821000005, 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:33.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895496672999997, 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:33.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895486066999993, 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:33.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895475728999998, 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:33.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:33.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:33.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:33.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.939 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895465721999997, 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:33.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895455771000002, 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:33.944 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:33.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.895440463, 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:33.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.954 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.893464576, 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:33.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.893453620000003, 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:33.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.893441131000003, 2024-09-17T21:47:33.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:33.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:33.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 76.893430667999993, 2024-09-17T21:47:33.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:33.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:33.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.974 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:33.974 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:33.974 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:33.975 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:47:33.976 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:33.972+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:33.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:34.045 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone32 from 0xe34ee00 (overlap 0x258ea00) to 0x21cd200 2024-09-17T21:47:34.078 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:34.073+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:34.096 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.097 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:34.097 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:34.097 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:34.106 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:34.190 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:34.188+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:34.254 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.255 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:34.255 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:34.255 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:34.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:34.277 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:34.275+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:34.378 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:34.376+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:34.412 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-17T21:47:34.417 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.417 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.730762720000001, 2024-09-17T21:47:34.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:34.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:34.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:34.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.698487744999994, 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:34.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:34.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:34.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.698248977000006, 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:34.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:34.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:34.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:34.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.553539362000002, 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:34.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:34.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:627 3.6 3:6244dd87:::rbd_data.1112bc1b2568.0000000000000019:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.463019629000001, 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030691610000000001, 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 627 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950565+0000", 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950567+0000", 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950574+0000", 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950580+0000", 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950631+0000", 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1863999999999997e-05 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950752+0000", 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120596 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981256+0000", 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030503834000000001 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:745 3.6 3:60bbb74c:::rbd_data.1112bc1b2568.00000000000000de:head [delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.455110603000001, 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023010439000000001, 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 745 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958474+0000", 2024-09-17T21:47:34.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958476+0000", 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2299999999999998e-06 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958482+0000", 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958488+0000", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961234+0000", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027465929999999999 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961306+0000", 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1521999999999997e-05 2024-09-17T21:47:34.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961496+0000", 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190053 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.962386+0000", 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089013799999999995 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:34.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981403+0000", 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019016943000000001 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981448+0000", 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5212e-05 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981484+0000", 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5608999999999997e-05 2024-09-17T21:47:34.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1235 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426554078999999, 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013460730000000001, 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1235 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987032+0000", 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9209999999999999e-06 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987043+0000", 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1076e-05 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987049+0000", 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000331+0000", 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013281866999999999 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000405+0000", 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4004000000000005e-05 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000491+0000", 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6062000000000004e-05 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1236 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426528149999999, 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887803000000001, 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1236 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:34.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987058+0000", 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987065+0000", 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1219999999999997e-06 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987070+0000", 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9529999999999998e-06 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000520+0000", 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013449796999999999 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000592+0000", 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919e-05 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000944+0000", 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035145700000000001 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1237 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426507043000001, 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049556, 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1237 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987080+0000", 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987086+0000", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6309999999999997e-06 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987091+0000", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000974+0000", 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013882864999999999 2024-09-17T21:47:34.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001045+0000", 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0661000000000007e-05 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001126+0000", 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1521999999999996e-05 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1264 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426336122999999, 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014065061, 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1264 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987249+0000", 2024-09-17T21:47:34.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987252+0000", 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987263+0000", 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1289e-05 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987270+0000", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1330000000000001e-06 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001154+0000", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01388413 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001225+0000", 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0975000000000004e-05 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001313+0000", 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7404999999999994e-05 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1265 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426302813, 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01421443, 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1265 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987282+0000", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987284+0000", 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.328e-06 2024-09-17T21:47:34.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987291+0000", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987305+0000", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3344e-05 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001341+0000", 2024-09-17T21:47:34.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014035903000000001 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001411+0000", 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0502999999999999e-05 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001496+0000", 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4383999999999997e-05 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1266 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426270873, 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014670829, 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1266 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:34.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987314+0000", 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987316+0000", 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4059999999999999e-06 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987323+0000", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987334+0000", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.093e-05 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001524+0000", 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014189715 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001592+0000", 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8132000000000004e-05 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001984+0000", 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039185199999999999 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1267 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426241903000001, 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014824201, 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1267 2024-09-17T21:47:34.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987343+0000", 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987345+0000", 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987351+0000", 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987356+0000", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002014+0000", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014657411 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002084+0000", 2024-09-17T21:47:34.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0596e-05 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002166+0000", 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2083999999999995e-05 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1268 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:34.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426220254, 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014992027999999999, 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1268 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987369+0000", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7129999999999996e-06 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987376+0000", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987381+0000", 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-17T21:47:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002195+0000", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014814226 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002268+0000", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3279000000000001e-05 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002356+0000", 2024-09-17T21:47:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7305999999999994e-05 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1269 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.426193090000002, 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015291325999999999, 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1269 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987394+0000", 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3989999999999998e-06 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987401+0000", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8210000000000004e-06 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987406+0000", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0030000000000002e-06 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002522+0000", 2024-09-17T21:47:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015116368 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002595+0000", 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3335000000000002e-05 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002682+0000", 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6927000000000003e-05 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:771 3.1 3:8006aab1:::rbd_data.1112f0bef54a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 8280576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.809440037000002, 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013777717, 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 771 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604145+0000", 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608094+0000", 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039492950000000002 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608114+0000", 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0254e-05 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608122+0000", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608205+0000", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3511999999999996e-05 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608304+0000", 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8524999999999996e-05 2024-09-17T21:47:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608621+0000", 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031733999999999999 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617430+0000", 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088086459999999998 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617787+0000", 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035710100000000002 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617854+0000", 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7699000000000004e-05 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617922+0000", 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7169999999999996e-05 2024-09-17T21:47:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.530742846999999, 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.523841076, 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.523724985000001, 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:34.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:34.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2165 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch ping cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.217489906000001, 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69677434100000002, 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2165 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:34.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196096+0000", 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196108+0000", 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196130+0000", 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1695000000000002e-05 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196140+0000", 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196216+0000", 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899e-05 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196227+0000", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1008e-05 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892718+0000", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69649070000000002 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892759+0000", 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1876999999999999e-05 2024-09-17T21:47:34.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892868+0000", 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010892599999999999 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.527 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:34.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:34.530 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:34.526+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:34.530 INFO:teuthology.orchestra.run.smithi165.stdout:4058 read 0x382d200 thru 0x382d9a3 (0x7a4 bytes) 2024-09-17T21:47:34.530 INFO:teuthology.orchestra.run.smithi165.stdout:4059 read 0x482c000 thru 0x482c90e (0x90f bytes) 2024-09-17T21:47:34.530 INFO:teuthology.orchestra.run.smithi165.stdout:4060 punch from 0x1ef4800 to 0x1ef9a00, (0x5200 bytes) 2024-09-17T21:47:34.530 INFO:teuthology.orchestra.run.smithi165.stdout:4062 write 0x6b2e000 thru 0x6b324c8 (0x44c9 bytes) 2024-09-17T21:47:34.531 INFO:teuthology.orchestra.run.smithi165.stdout:4063 read 0xc689c00 thru 0xc693dee (0xa1ef bytes) 2024-09-17T21:47:34.531 INFO:teuthology.orchestra.run.smithi165.stdout:4065 write 0x5c8d400 thru 0x5c98a1c (0xb61d bytes) 2024-09-17T21:47:34.531 INFO:teuthology.orchestra.run.smithi165.stdout:4066 read 0x4ae7e00 thru 0x4aea472 (0x2673 bytes) 2024-09-17T21:47:34.531 INFO:teuthology.orchestra.run.smithi165.stdout:4067 read 0xbdc1400 thru 0xbdcfe07 (0xea08 bytes) 2024-09-17T21:47:34.531 INFO:teuthology.orchestra.run.smithi165.stdout:4068 trunc from 0xe34ee00 to 0x5b1e800 2024-09-17T21:47:34.629 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:34.627+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:34.678 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:34.678 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:34.679 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:34.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:34.730 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:34.728+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:34.806 INFO:teuthology.orchestra.run.smithi165.stdout:4070 write 0xb375000 thru 0xb383ef9 (0xeefa bytes) 2024-09-17T21:47:34.825 INFO:teuthology.orchestra.run.smithi165.stdout:4071 write 0xed8a400 thru 0xed8cebd (0x2abe bytes) 2024-09-17T21:47:34.825 INFO:teuthology.orchestra.run.smithi165.stdout:4072 write 0xcf0e200 thru 0xcf13826 (0x5627 bytes) 2024-09-17T21:47:34.826 INFO:teuthology.orchestra.run.smithi165.stdout:4074 write 0xc1b7200 thru 0xc1bdcb8 (0x6ab9 bytes) 2024-09-17T21:47:34.826 INFO:teuthology.orchestra.run.smithi165.stdout:4075 flatten 2024-09-17T21:47:34.831 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:34.829+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:34.842 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:34.843 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:34.843 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:34.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:34.910 INFO:teuthology.orchestra.run.smithi165.stdout:4076 write 0x28a9a00 thru 0x28b5494 (0xba95 bytes) 2024-09-17T21:47:34.924 INFO:teuthology.orchestra.run.smithi165.stdout:4077 trunc from 0xed8d000 to 0x277fc00 2024-09-17T21:47:34.931 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:34.929+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:34.944 INFO:teuthology.orchestra.run.smithi165.stdout:4080 trunc from 0x277fc00 to 0x2aaf400 2024-09-17T21:47:34.968 INFO:teuthology.orchestra.run.smithi165.stdout:4081 trunc from 0x2aaf400 to 0xcc93a00 2024-09-17T21:47:34.976 INFO:teuthology.orchestra.run.smithi053.stdout:get_pg_num_history pg_num_history pg_num_history(e95 pg_nums {1={13=1},2={16=8},3={21=32}} deleted_pools ) 2024-09-17T21:47:34.976 INFO:teuthology.orchestra.run.smithi053.stdout:Importing pgid 3.a 2024-09-17T21:47:34.976 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:50216c96:::rbd_data.1112184ea9c8.0000000000000077:head# 2024-09-17T21:47:34.976 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 6=[]:{} 2024-09-17T21:47:34.977 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:5084e2dd:::rbd_data.1112d625dbc0.0000000000000042:1e# 2024-09-17T21:47:34.978 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:5084e2dd:::rbd_data.1112d625dbc0.0000000000000042:head# 2024-09-17T21:47:34.978 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 1e=[]:{1e=[1e]} 2024-09-17T21:47:34.979 INFO:teuthology.orchestra.run.smithi165.stdout:4083 write 0xb011c00 thru 0xb01a253 (0x8654 bytes) 2024-09-17T21:47:34.979 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:50a54057:::rbd_data.11121f8e87bb.0000000000000125:head# 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:510afc60:::rbd_data.11121f8e87bb.0000000000000008:head# 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:52c4a3ad:::rbd_data.1112bc1b2568.0000000000000004:head# 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 1a=[]:{} 2024-09-17T21:47:34.980 INFO:teuthology.orchestra.run.smithi165.stdout:4085 punch from 0x63b9400 to 0x63bbc00, (0x2800 bytes) 2024-09-17T21:47:34.981 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:52c7d6d0:::rbd_data.11128c113d55.0000000000000011:head# 2024-09-17T21:47:34.981 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:52f2ba4c:::rbd_data.1112a93a0526.0000000000000002:head# 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:535385f0:::rbd_children:head# 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:539e9b35:::rbd_data.11128c113d55.0000000000000004:head# 2024-09-17T21:47:34.982 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.983 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:545786ae:::rbd_id.image_client.0-clone13:head# 2024-09-17T21:47:34.983 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.983 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:54f8743c:::rbd_data.1112fae1757a.0000000000000130:1f# 2024-09-17T21:47:34.984 INFO:teuthology.orchestra.run.smithi165.stdout:4088 read 0x14ea200 thru 0x14eaf9e (0xd9f bytes) 2024-09-17T21:47:34.984 INFO:teuthology.orchestra.run.smithi165.stdout:4089 punch from 0x9dd1600 to 0x9dd3200, (0x1c00 bytes) 2024-09-17T21:47:34.984 INFO:teuthology.orchestra.run.smithi165.stdout:4091 read 0x7db400 thru 0x7e0009 (0x4c0a bytes) 2024-09-17T21:47:34.984 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:54f8743c:::rbd_data.1112fae1757a.0000000000000130:head# 2024-09-17T21:47:34.984 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 1f=[]:{1f=[1f]} 2024-09-17T21:47:34.985 INFO:teuthology.orchestra.run.smithi165.stdout:4092 punch from 0xa54e00 to 0xa5b200, (0x6400 bytes) 2024-09-17T21:47:34.985 INFO:teuthology.orchestra.run.smithi165.stdout:4093 read 0xc39dc00 thru 0xc39e7c4 (0xbc5 bytes) 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:55056370:::rbd_data.11124390a7c3.000000000000000d:head# 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:556d2800:::rbd_data.11121f8e87bb.000000000000003c:head# 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:559e4506:::rbd_data.11121f8e87bb.0000000000000123:head# 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.986 INFO:teuthology.orchestra.run.smithi165.stdout:4095 trunc from 0xcc93a00 to 0xb5b0200 2024-09-17T21:47:34.987 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:56770158:::rbd_data.11121f8e87bb.0000000000000091:head# 2024-09-17T21:47:34.987 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:34.988 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:568819d6:::rbd_data.1112e01b91a8.0000000000000017:9# 2024-09-17T21:47:34.988 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:34.988 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080737490999994, 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:34.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080727190000005, 2024-09-17T21:47:34.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:34.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:34.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:34.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080711116000003, 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:34.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080699902999996, 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:35.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080682464999995, 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:35.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080671594999998, 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:35.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:35.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:35.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.080476718, 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:35.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:35.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:35.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.078772048000005, 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:35.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.078761409999998, 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.078751316999998, 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.078741198000003, 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.076610967999997, 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.076601237000006, 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:35.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:35.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.075817877999995, 2024-09-17T21:47:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:35.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:35.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:35.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.075802307999993, 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:35.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.075709406000001, 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:35.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:35.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.075214786999993, 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:35.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:35.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.075070741999994, 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:35.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.074385305000007, 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:35.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.074022915, 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:35.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:35.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.094 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:35.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:35.098 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:35.093+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:35.098 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:568819d6:::rbd_data.1112e01b91a8.0000000000000017:head# 2024-09-17T21:47:35.098 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 9=[]:{9=[9]} 2024-09-17T21:47:35.098 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:56921cc9:::rbd_data.11121f8e87bb.000000000000005f:head# 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:569eea4d:::rbd_data.1112184ea9c8.0000000000000076:head# 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 6=[]:{} 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:Write #3:57403a42:::rbd_object_map.1112a8bc19be.000000000000000b:head# 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:snapset 0=[]:{} 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:write_pg epoch 95 info 3.a( v 95'186 (0'0,95'186] local-lis/les=80/81 n=22 ec=21/21 lis/c=80/80 les/c/f=81/81/0 sis=80) 2024-09-17T21:47:35.099 INFO:teuthology.orchestra.run.smithi053.stdout:Import successful 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4097 punch from 0x48a1400 to 0x48a8400, (0x7000 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4099 punch from 0x8f3c200 to 0x8f45c00, (0x9a00 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4100 read 0xb2e6600 thru 0xb2e8236 (0x1c37 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4102 write 0x8a18400 thru 0x8a1919c (0xd9d bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4103 write 0x5dbb800 thru 0x5dbf858 (0x4059 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4105 read 0x2510800 thru 0x251d942 (0xd143 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4106 read 0x4474600 thru 0x447cd12 (0x8713 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4107 write 0xc200e00 thru 0xc20fb1a (0xed1b bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4108 punch from 0xb761600 to 0xb765400, (0x3e00 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4109 punch from 0x1066e00 to 0x106d200, (0x6400 bytes) 2024-09-17T21:47:35.100 INFO:teuthology.orchestra.run.smithi165.stdout:4111 write 0x87e4e00 thru 0x87f1aad (0xccae bytes) 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4113 trunc from 0xc20fc00 to 0xae8d800 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4114 trunc from 0xae8d800 to 0x32ec400 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4115 write 0x2738200 thru 0x27428d0 (0xa6d1 bytes) 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4118 write 0x3ff4e00 thru 0x4002237 (0xd438 bytes) 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4119 read 0x3bb5e00 thru 0x3bc080f (0xaa10 bytes) 2024-09-17T21:47:35.101 INFO:teuthology.orchestra.run.smithi165.stdout:4122 write 0x48bde00 thru 0x48cdabe (0xfcbf bytes) 2024-09-17T21:47:35.111 INFO:teuthology.orchestra.run.smithi165.stdout:4123 trunc from 0x48cdc00 to 0xbf84e00 2024-09-17T21:47:35.123 INFO:teuthology.orchestra.run.smithi165.stdout:4124 clone 34 order 22 su 524288 sc 8 2024-09-17T21:47:35.196 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:35.194+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:35.244 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:35.244 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:35.245 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:35.245 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:35.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:35.298 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:35.295+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:35.360 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:35.360 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:35.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:35.399 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:35.397+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4542.0:1227 3.1e 3:7912a5e4:::rbd_data.11125c054388.0000000000000007:head [stat,truncate 10485760] snapc 7=[7] ondisk+write+known_if_redirected+supports_pool_eio e30)", 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.663982220999998, 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010410895999999999, 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4542", 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2446051467", 2024-09-17T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1227 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809924+0000", 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809926+0000", 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.686e-06 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809933+0000", 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2849999999999997e-06 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809951+0000", 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7448999999999998e-05 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.809958+0000", 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3130000000000001e-06 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810009+0000", 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1372000000000002e-05 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810061+0000", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1705e-05 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.810166+0000", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001052 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.814688+0000", 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045213980000000003 2024-09-17T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820263+0000", 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005575287 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820298+0000", 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5355999999999998e-05 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:11.820335+0000", 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6844999999999997e-05 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:900 3.8 3:123b94fd:::rbd_data.1112a8bc19be.0000000000000007:head [stat,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.654331257000003, 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010632442000000001, 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 900 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819575+0000", 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819576+0000", 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9299999999999996e-07 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819580+0000", 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819589+0000", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4900000000000006e-06 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819597+0000", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0630000000000006e-06 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819659+0000", 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1928999999999996e-05 2024-09-17T21:47:35.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.819786+0000", 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000126866 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.830208+0000", 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010421332 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4122 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.570385387999998, 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013435351, 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4122 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903521+0000", 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.903522+0000", 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.125e-06 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904937+0000", 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014151050000000001 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904956+0000", 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9264000000000001e-05 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904964+0000", 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9400000000000002e-06 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905042+0000", 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7510999999999997e-05 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905183+0000", 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014138000000000001 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.905542+0000", 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035897400000000002 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908895+0000", 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003352945 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916848+0000", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079531289999999998 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916913+0000", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4677000000000004e-05 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916956+0000", 2024-09-17T21:47:35.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300999999999998e-05 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4123 3.1e 3:7dd12bda:::rbd_data.1112a8bc19be.000000000000000e:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.568923253000001, 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012100282, 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4123 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904983+0000", 2024-09-17T21:47:35.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904984+0000", 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7100000000000001e-07 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.904987+0000", 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0290000000000001e-06 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907497+0000", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0025097660000000001 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907510+0000", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3498e-05 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907565+0000", 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4939999999999999e-05 2024-09-17T21:47:35.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907671+0000", 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000105643 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907863+0000", 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019272500000000001 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910290+0000", 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002426433 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.916998+0000", 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0067076610000000002 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917048+0000", 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9976999999999999e-05 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917083+0000", 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5839e-05 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4124 3.18 3:1cfc1be9:::rbd_data.1112a8bc19be.000000000000000c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.566312134999997, 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010043765, 2024-09-17T21:47:35.501 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4124 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907594+0000", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.907596+0000", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9609999999999999e-06 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908285+0000", 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068860099999999995 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908296+0000", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0878e-05 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.908303+0000", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8650000000000003e-06 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909138+0000", 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00083564000000000002 2024-09-17T21:47:35.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.909454+0000", 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031576500000000002 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.910059+0000", 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00060513299999999999 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917507+0000", 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074480950000000001 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917545+0000", 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7732999999999999e-05 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917598+0000", 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2797999999999998e-05 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.917638+0000", 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0296000000000002e-05 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4841 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:35.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.502197678000002, 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010021983999999999, 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4841 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971709+0000", 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.971710+0000", 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0759999999999999e-06 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972398+0000", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00068840399999999999 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972411+0000", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3016e-05 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972419+0000", 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5750000000000003e-06 2024-09-17T21:47:35.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972467+0000", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8489999999999998e-05 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972552+0000", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4361000000000001e-05 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972856+0000", 2024-09-17T21:47:35.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000304653 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.979169+0000", 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0063128430000000003 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981640+0000", 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002471025 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981689+0000", 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8967000000000001e-05 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981731+0000", 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1573999999999999e-05 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4720.0:4842 3.7 3:e291a785:::rbd_data.1112a8bc19be.000000000000001b:head [delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.501469456000002, 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011399714, 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4720", 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1934619487", 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4842 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972437+0000", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972438+0000", 2024-09-17T21:47:35.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6799999999999999e-07 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972441+0000", 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.123e-06 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972448+0000", 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2189999999999998e-06 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.972453+0000", 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3709999999999999e-06 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974167+0000", 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001714042 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974239+0000", 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1779000000000005e-05 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.974509+0000", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026969800000000002 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.981770+0000", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0072614669999999997 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983751+0000", 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019804029999999999 2024-09-17T21:47:35.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983800+0000", 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8789999999999999e-05 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:31.983837+0000", 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7054000000000001e-05 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:47:35.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 61.16789232, 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.601133818000001, 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.593611676000002, 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:47:35.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1128 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.483322604000001, 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099832400000000009, 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1128 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990584+0000", 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990586+0000", 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.125e-06 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990593+0000", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6919999999999996e-06 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990598+0000", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9830000000000004e-06 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000414+0000", 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098157309999999994 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000481+0000", 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7303000000000003e-05 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000567+0000", 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5923999999999999e-05 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1129 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 2031616~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.483302052999999, 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010143592999999999, 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1129 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990604+0000", 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990605+0000", 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990607+0000", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0209999999999998e-06 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990613+0000", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3640000000000004e-06 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.990618+0000", 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7859999999999996e-06 2024-09-17T21:47:35.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000596+0000", 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099777809999999998 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000662+0000", 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5891999999999999e-05 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000748+0000", 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6372999999999994e-05 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1224 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.47645262, 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010061038, 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1224 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997454+0000", 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.997457+0000", 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3499999999999999e-06 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998930+0000", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0014735079999999999 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.998947+0000", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6977999999999999e-05 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007338+0000", 2024-09-17T21:47:35.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083907679999999998 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007417+0000", 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9008e-05 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.007515+0000", 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7958e-05 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1255 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.474476708000001, 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010112939, 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1255 2024-09-17T21:47:35.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999430+0000", 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999432+0000", 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249999999999999e-06 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999439+0000", 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9569999999999996e-06 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999445+0000", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2730000000000002e-06 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009368+0000", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099234240000000001 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009446+0000", 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8209999999999998e-05 2024-09-17T21:47:35.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009543+0000", 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6353000000000004e-05 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1256 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1835008~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.474455243000001, 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010198560000000001, 2024-09-17T21:47:35.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1256 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999451+0000", 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999452+0000", 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:35.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999453+0000", 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.945e-06 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999460+0000", 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7789999999999998e-06 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.999469+0000", 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6309999999999992e-06 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009556+0000", 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010086927000000001 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009597+0000", 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1038999999999998e-05 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.009650+0000", 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2831999999999998e-05 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1288 3.1e 3:7ff55869:::rbd_data.111297114b95.0000000000000003:19 [read 1867776~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.471438924000001, 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010050948000000001, 2024-09-17T21:47:35.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1288 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002467+0000", 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002468+0000", 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002470+0000", 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.384e-06 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002477+0000", 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002482+0000", 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1089999999999997e-06 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012369+0000", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0098866030000000008 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012434+0000", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5512999999999994e-05 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012518+0000", 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4089000000000001e-05 2024-09-17T21:47:35.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1307 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.463421671999999, 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099939750000000004, 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1307 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010485+0000", 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010488+0000", 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0240000000000002e-06 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010495+0000", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010508+0000", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3410999999999999e-05 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017483+0000", 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0069745320000000003 2024-09-17T21:47:35.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017570+0000", 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7057000000000004e-05 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.017970+0000", 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00040009600000000001 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020303+0000", 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023332729999999999 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020439+0000", 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000135783 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020462+0000", 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2469000000000001e-05 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020479+0000", 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7144999999999999e-05 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1309 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.463389275000001, 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010826840000000001, 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1309 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010517+0000", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010518+0000", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0299999999999999e-07 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010521+0000", 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.207e-06 2024-09-17T21:47:35.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010528+0000", 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0709999999999997e-06 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010533+0000", 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1930000000000001e-06 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018747+0000", 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082139880000000002 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.018835+0000", 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7560999999999997e-05 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.019193+0000", 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035790399999999998 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020506+0000", 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0013139099999999999 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021255+0000", 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00074901999999999998 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021305+0000", 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9230999999999997e-05 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021344+0000", 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9252000000000003e-05 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1311 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.463365781, 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012406011, 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1311 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010541+0000", 2024-09-17T21:47:35.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010544+0000", 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.977e-06 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010551+0000", 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7240000000000003e-06 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.010556+0000", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0989999999999998e-06 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020524+0000", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.009968457 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020568+0000", 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4132999999999999e-05 2024-09-17T21:47:35.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.020784+0000", 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00021525299999999999 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022838+0000", 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0020542030000000001 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022848+0000", 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9879999999999999e-06 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022894+0000", 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5803000000000002e-05 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022947+0000", 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2896999999999999e-05 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1312 3.18 3:1a519963:::rbd_object_map.1112bc1b2568:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.460983228, 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011273178999999999, 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:35.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1312 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012923+0000", 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012924+0000", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2309999999999999e-06 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012937+0000", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2662e-05 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012950+0000", 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.324e-05 2024-09-17T21:47:35.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.012958+0000", 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9140000000000008e-06 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021388+0000", 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084298759999999993 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:35.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021609+0000", 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000221256 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.021953+0000", 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00034352200000000002 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.022955+0000", 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0010025909999999999 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024087+0000", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0011314109999999999 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024148+0000", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1001000000000001e-05 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.024196+0000", 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8474999999999999e-05 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:35.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:35.587 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:35.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:35.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f /var/log/ceph/exp.3.a.3 2024-09-17T21:47:35.590 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:35.590 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:35.586+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:35.703 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:35.701+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:35.751 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:35.751 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:35.751 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:35.752 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:35.752 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:35.752 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:35.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:35.769 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:35.769 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:35.786 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:35.786 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:35.801 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:35.799+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:35.803 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:35.804 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:35.820 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:35.820 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:35.824 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:35.890 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:35.888+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:35.909 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:35.909 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:35.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:35.979 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:47:36.024 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:36.022+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:36.029 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.029 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:36.092 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:36.089+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:36.149 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.149 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:36.192 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:36.190+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:36.293 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:36.290+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:36.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.316 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:36.394 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:36.392+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:36.421 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-17T21:47:36.482 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.482 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.493 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:36.494 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:36.492+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:36.595 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:36.593+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:36.614 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.614 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:36.638 INFO:teuthology.orchestra.run.smithi165.stdout:4125 trunc from 0xbf84e00 to 0xbc18800 2024-09-17T21:47:36.660 INFO:teuthology.orchestra.run.smithi165.stdout:4126 punch from 0x79cf600 to 0x79da600, (0xb000 bytes) 2024-09-17T21:47:36.662 INFO:teuthology.orchestra.run.smithi165.stdout:4129 read 0x9ebbe00 thru 0x9ec529d (0x949e bytes) 2024-09-17T21:47:36.674 INFO:teuthology.orchestra.run.smithi165.stdout:4130 write 0xd5a4400 thru 0xd5adc85 (0x9886 bytes) 2024-09-17T21:47:36.676 INFO:teuthology.orchestra.run.smithi165.stdout:4132 read 0x2737e00 thru 0x2742450 (0xa651 bytes) 2024-09-17T21:47:36.677 INFO:teuthology.orchestra.run.smithi165.stdout:4133 read 0xa0bdc00 thru 0xa0cd2fc (0xf6fd bytes) 2024-09-17T21:47:36.679 INFO:teuthology.orchestra.run.smithi165.stdout:4134 trunc from 0xd5ade00 to 0xec45e00 2024-09-17T21:47:36.690 INFO:teuthology.orchestra.run.smithi165.stdout:4135 write 0x43d4800 thru 0x43de837 (0xa038 bytes) 2024-09-17T21:47:36.691 INFO:teuthology.orchestra.run.smithi165.stdout:4137 write 0x8e57400 thru 0x8e618e0 (0xa4e1 bytes) 2024-09-17T21:47:36.691 INFO:teuthology.orchestra.run.smithi165.stdout:4138 write 0xd9a5600 thru 0xd9ae8bd (0x92be bytes) 2024-09-17T21:47:36.691 INFO:teuthology.orchestra.run.smithi165.stdout:4139 punch from 0xcde9000 to 0xcdf1800, (0x8800 bytes) 2024-09-17T21:47:36.692 INFO:teuthology.orchestra.run.smithi165.stdout:4140 read 0x1049800 thru 0x104d9e6 (0x41e7 bytes) 2024-09-17T21:47:36.693 INFO:teuthology.orchestra.run.smithi165.stdout:4141 write 0xa6e8c00 thru 0xa6f32c3 (0xa6c4 bytes) 2024-09-17T21:47:36.693 INFO:teuthology.orchestra.run.smithi165.stdout:4142 write 0xddd0600 thru 0xdddcbfc (0xc5fd bytes) 2024-09-17T21:47:36.693 INFO:teuthology.orchestra.run.smithi165.stdout:4144 punch from 0xa7e6400 to 0xa7e6800, (0x400 bytes) 2024-09-17T21:47:36.694 INFO:teuthology.orchestra.run.smithi165.stdout:4145 read 0xb89c00 thru 0xb8e4e3 (0x48e4 bytes) 2024-09-17T21:47:36.695 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:36.693+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:36.707 INFO:teuthology.orchestra.run.smithi165.stdout:4146 trunc from 0xec45e00 to 0x3f5ac00 2024-09-17T21:47:36.729 INFO:teuthology.orchestra.run.smithi165.stdout:4149 read 0x2a1b200 thru 0x2a28850 (0xd651 bytes) 2024-09-17T21:47:36.729 INFO:teuthology.orchestra.run.smithi165.stdout:4151 read 0x2b6b200 thru 0x2b7a0a9 (0xeeaa bytes) 2024-09-17T21:47:36.730 INFO:teuthology.orchestra.run.smithi165.stdout:4154 read 0x3286400 thru 0x328e883 (0x8484 bytes) 2024-09-17T21:47:36.730 INFO:teuthology.orchestra.run.smithi165.stdout:4156 read 0x19f0e00 thru 0x19f4b4b (0x3d4c bytes) 2024-09-17T21:47:36.741 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.741 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.744 INFO:teuthology.orchestra.run.smithi165.stdout:4159 write 0x80f1400 thru 0x80f43ef (0x2ff0 bytes) 2024-09-17T21:47:36.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:36.752 INFO:teuthology.orchestra.run.smithi165.stdout:4161 trunc from 0x80f4400 to 0x25bf600 2024-09-17T21:47:36.786 INFO:teuthology.orchestra.run.smithi165.stdout:4165 write 0x73f9200 thru 0x73faf5a (0x1d5b bytes) 2024-09-17T21:47:36.787 INFO:teuthology.orchestra.run.smithi165.stdout:4166 read 0xef1600 thru 0xef9f84 (0x8985 bytes) 2024-09-17T21:47:36.793 INFO:teuthology.orchestra.run.smithi165.stdout:4167 trunc from 0x73fb000 to 0xca48600 2024-09-17T21:47:36.821 INFO:teuthology.orchestra.run.smithi165.stdout:4169 write 0xdd87200 thru 0xdd87a2f (0x830 bytes) 2024-09-17T21:47:36.822 INFO:teuthology.orchestra.run.smithi165.stdout:4170 punch from 0x4aed800 to 0x4aeee00, (0x1600 bytes) 2024-09-17T21:47:36.832 INFO:teuthology.orchestra.run.smithi165.stdout:4172 trunc from 0xdd87c00 to 0x63d4400 2024-09-17T21:47:36.834 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:36.832+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:36.847 INFO:teuthology.orchestra.run.smithi165.stdout:4175 punch from 0x2cc6600 to 0x2cd1600, (0xb000 bytes) 2024-09-17T21:47:36.855 INFO:teuthology.orchestra.run.smithi165.stdout:4176 trunc from 0x63d4400 to 0x23f9000 2024-09-17T21:47:36.869 INFO:teuthology.orchestra.run.smithi165.stdout:4178 write 0x12c2e00 thru 0x12c5868 (0x2a69 bytes) 2024-09-17T21:47:36.876 INFO:teuthology.orchestra.run.smithi165.stdout:4179 punch from 0x525200 to 0x52de00, (0x8c00 bytes) 2024-09-17T21:47:36.878 INFO:teuthology.orchestra.run.smithi165.stdout:4180 read 0x1364e00 thru 0x136b067 (0x6268 bytes) 2024-09-17T21:47:36.879 INFO:teuthology.orchestra.run.smithi165.stdout:4183 punch from 0x1cf5600 to 0x1cfde00, (0x8800 bytes) 2024-09-17T21:47:36.879 INFO:teuthology.orchestra.run.smithi165.stdout:4184 read 0x1e2a200 thru 0x1e3921f (0xf020 bytes) 2024-09-17T21:47:36.879 INFO:teuthology.orchestra.run.smithi165.stdout:4185 punch from 0x71600 to 0x7ea00, (0xd400 bytes) 2024-09-17T21:47:36.897 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:36.895+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:36.904 INFO:teuthology.orchestra.run.smithi165.stdout:4187 write 0xc72d400 thru 0xc73c418 (0xf019 bytes) 2024-09-17T21:47:36.905 INFO:teuthology.orchestra.run.smithi165.stdout:4189 read 0x32f0c00 thru 0x330037d (0xf77e bytes) 2024-09-17T21:47:36.905 INFO:teuthology.orchestra.run.smithi165.stdout:4191 read 0x3f5ce00 thru 0x3f5d15e (0x35f bytes) 2024-09-17T21:47:36.906 INFO:teuthology.orchestra.run.smithi165.stdout:4195 read 0x62400 thru 0x6281e (0x41f bytes) 2024-09-17T21:47:36.909 INFO:teuthology.orchestra.run.smithi165.stdout:4196 read 0x2eb2000 thru 0x2eb9fdd (0x7fde bytes) 2024-09-17T21:47:36.910 INFO:teuthology.orchestra.run.smithi165.stdout:4197 read 0xba4e00 thru 0xba783f (0x2a40 bytes) 2024-09-17T21:47:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:4198 read 0x794ba00 thru 0x79537f6 (0x7df7 bytes) 2024-09-17T21:47:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:4200 read 0x7e5d400 thru 0x7e6754b (0xa14c bytes) 2024-09-17T21:47:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:4202 read 0x99f6c00 thru 0x9a029f3 (0xbdf4 bytes) 2024-09-17T21:47:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:4206 punch from 0x490d200 to 0x490f400, (0x2200 bytes) 2024-09-17T21:47:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:4207 flatten 2024-09-17T21:47:36.915 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:36.915 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:36.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:36.937 INFO:teuthology.orchestra.run.smithi165.stdout:4209 read 0x2a74c00 thru 0x2a78950 (0x3d51 bytes) 2024-09-17T21:47:36.939 INFO:teuthology.orchestra.run.smithi165.stdout:4211 punch from 0x144e800 to 0x1457600, (0x8e00 bytes) 2024-09-17T21:47:36.939 INFO:teuthology.orchestra.run.smithi165.stdout:4212 write 0x6906a00 thru 0x690a6ea (0x3ceb bytes) 2024-09-17T21:47:36.939 INFO:teuthology.orchestra.run.smithi165.stdout:4214 punch from 0xa5a4e00 to 0xa5b2c00, (0xde00 bytes) 2024-09-17T21:47:36.950 INFO:teuthology.orchestra.run.smithi165.stdout:4215 trunc from 0xc73c600 to 0x2e37a00 2024-09-17T21:47:36.974 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:47:36.994 INFO:teuthology.orchestra.run.smithi165.stdout:4216 write 0x88a8600 thru 0x88acb6e (0x456f bytes) 2024-09-17T21:47:36.998 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:36.996+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:37.000 INFO:teuthology.orchestra.run.smithi165.stdout:4217 trunc from 0x88acc00 to 0xd5a1400 2024-09-17T21:47:37.017 INFO:teuthology.orchestra.run.smithi165.stdout:4221 read 0x434e400 thru 0x43581fd (0x9dfe bytes) 2024-09-17T21:47:37.034 INFO:teuthology.orchestra.run.smithi165.stdout:4222 write 0xebf7e00 thru 0xec01be7 (0x9de8 bytes) 2024-09-17T21:47:37.035 INFO:teuthology.orchestra.run.smithi165.stdout:4227 write 0xe45ee00 thru 0xe46d9cb (0xebcc bytes) 2024-09-17T21:47:37.035 INFO:teuthology.orchestra.run.smithi165.stdout:4229 read 0x8343600 thru 0x834aea5 (0x78a6 bytes) 2024-09-17T21:47:37.035 INFO:teuthology.orchestra.run.smithi165.stdout:4230 write 0x28bb400 thru 0x28bfec6 (0x4ac7 bytes) 2024-09-17T21:47:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:4231 write 0x5675800 thru 0x5683767 (0xdf68 bytes) 2024-09-17T21:47:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:4232 read 0xb107a00 thru 0xb107b8d (0x18e bytes) 2024-09-17T21:47:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:4233 write 0xe455e00 thru 0xe45c59e (0x679f bytes) 2024-09-17T21:47:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:4235 write 0x13ae800 thru 0x13bbc1b (0xd41c bytes) 2024-09-17T21:47:37.036 INFO:teuthology.orchestra.run.smithi165.stdout:4239 clone 35 order 19 su 2048 sc 5 2024-09-17T21:47:37.086 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.086 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:37.138 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:37.136+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:37.199 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:37.197+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:37.301 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:37.299+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:37.357 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.357 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.365 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:37.402 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:37.400+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:37.472 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.472 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:37.524 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:37.522+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:37.587 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.587 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:37.636 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:37.634+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:37.704 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:37.702+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:37.742 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.742 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:37.805 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:37.802+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:37.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:37.872 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:37.881 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:37.905 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:37.903+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:37.948 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:37.948 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:37.965 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:37.965 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:37.986 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:37.986 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:38.004 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:38.004 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:38.006 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:38.004+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:38.023 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:38.023 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:38.026 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:38.027 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:38.032 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:38.033 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:38.033 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:38.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:38.107 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:38.105+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.195197993999997, 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.181211938000004, 2024-09-17T21:47:38.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:38.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:38.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.179250730999996, 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:38.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:38.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:38.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:38.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.177217373999994, 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:38.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.175239456, 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.175216661999997, 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:38.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:38.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:38.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.169307599000007, 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.167276540000003, 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:38.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.167249966, 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:38.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.166640173000005, 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:38.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.166622587000006, 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:38.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.166607630000001, 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:38.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:38.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.450926142, 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:38.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:38.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:38.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:38.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.450895521000007, 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.450870902999995, 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:38.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:38.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.450848956000002, 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:38.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:38.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.238 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 80.450827485999994, 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:38.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:38.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.409410683000004, 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.408366223000002, 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:38.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:38.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.207180892, 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:38.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:38.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.262 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:38.264 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:38.264 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:38.261+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:38.364 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:38.361+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:38.375 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:38.375 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:38.375 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:38.375 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:38.384 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:38.468 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:38.465+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:38.540 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:38.580 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:38.577+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:38.637 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-17T21:47:38.667 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:38.664+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:38.692 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:38.692 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:38.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718342914999994, 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:38.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718320422000005, 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:38.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718296162000001, 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:38.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:38.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718270278999995, 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:38.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718245277999998, 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:38.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:38.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:38.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718205479000005, 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:38.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:38.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:38.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718189734000006, 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:38.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:38.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:38.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718178198999993, 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:38.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.718167898999994, 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:38.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716196655999994, 2024-09-17T21:47:38.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:38.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716183508, 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:38.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:38.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716172901999997, 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:38.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716162564000001, 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716152557000001, 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716142606000005, 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:38.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:38.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.716127298000004, 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.714151411000003, 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:38.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:38.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:38.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.714140455000006, 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:38.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:38.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.714127966000007, 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:38.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.714117502999997, 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.779 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:38.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:38.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:38.783 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:38.785 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:38.786 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:38.782+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:38.885 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:38.883+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:38.909 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:38.910 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:38.910 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:38.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:38.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:47:39.004 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:39.002+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:39.071 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.071 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:39.072 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:39.072 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:39.087 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:39.084+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:39.179 INFO:teuthology.orchestra.run.smithi165.stdout:4246 clone 36 order 21 su 8192 sc 13 2024-09-17T21:47:39.188 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:39.186+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:39.203 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.203 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.517547424, 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:39.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.485272449000007, 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.485033681000004, 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:9018 3.3 3.21c9c183 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e65)", 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.340324066000001, 2024-09-17T21:47:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79388020400000003, 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 9018 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860045+0000", 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860048+0000", 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0299999999999998e-06 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860061+0000", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3767e-05 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860094+0000", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2295999999999997e-05 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860116+0000", 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2062999999999999e-05 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860205+0000", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9188000000000001e-05 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:52.860212+0000", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2259999999999999e-06 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653909+0000", 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.79369678499999996 2024-09-17T21:47:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:53.653925+0000", 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5849e-05 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:627 3.6 3:6244dd87:::rbd_data.1112bc1b2568.0000000000000019:head [stat,delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.249804333, 2024-09-17T21:47:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030691610000000001, 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 627 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950564+0000", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950565+0000", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950567+0000", 2024-09-17T21:47:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950574+0000", 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7839999999999998e-06 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950580+0000", 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5110000000000003e-06 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950631+0000", 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1863999999999997e-05 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.950752+0000", 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120596 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981256+0000", 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.030503834000000001 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:745 3.6 3:60bbb74c:::rbd_data.1112bc1b2568.00000000000000de:head [delete] snapc 1a=[1a] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.241895307, 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023010439000000001, 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 745 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958473+0000", 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958474+0000", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958476+0000", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2299999999999998e-06 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958482+0000", 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4119999999999998e-06 2024-09-17T21:47:39.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.958488+0000", 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3070000000000002e-06 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961234+0000", 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0027465929999999999 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961306+0000", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1521999999999997e-05 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.961496+0000", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190053 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.962386+0000", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00089013799999999995 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:39.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981403+0000", 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019016943000000001 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981448+0000", 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5212e-05 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.981484+0000", 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5608999999999997e-05 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1235 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213338782999998, 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013460730000000001, 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.236 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1235 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987030+0000", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987032+0000", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9209999999999999e-06 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987043+0000", 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1076e-05 2024-09-17T21:47:39.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987049+0000", 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4020000000000001e-06 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000331+0000", 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013281866999999999 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000405+0000", 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4004000000000005e-05 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000491+0000", 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6062000000000004e-05 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1236 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1572864~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.240 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213312854000002, 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887803000000001, 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1236 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987056+0000", 2024-09-17T21:47:39.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987058+0000", 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.114e-06 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987065+0000", 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1219999999999997e-06 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987070+0000", 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9529999999999998e-06 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000520+0000", 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013449796999999999 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000592+0000", 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919e-05 2024-09-17T21:47:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000944+0000", 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035145700000000001 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1237 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1900544~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213291747, 2024-09-17T21:47:39.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014049556, 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1237 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987077+0000", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987080+0000", 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987086+0000", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6309999999999997e-06 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987091+0000", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1590000000000001e-06 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.000974+0000", 2024-09-17T21:47:39.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013882864999999999 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001045+0000", 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0661000000000007e-05 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001126+0000", 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1521999999999996e-05 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1264 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 294912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213120826999997, 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014065061, 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1264 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987248+0000", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987249+0000", 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2099999999999996e-07 2024-09-17T21:47:39.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987252+0000", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987263+0000", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1289e-05 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987270+0000", 2024-09-17T21:47:39.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1330000000000001e-06 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001154+0000", 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01388413 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001225+0000", 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0975000000000004e-05 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001313+0000", 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7404999999999994e-05 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1265 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 622592~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213087516999998, 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01421443, 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1265 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987281+0000", 2024-09-17T21:47:39.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987282+0000", 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987284+0000", 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.328e-06 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987291+0000", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4410000000000004e-06 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987305+0000", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3344e-05 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001341+0000", 2024-09-17T21:47:39.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014035903000000001 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001411+0000", 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0502999999999999e-05 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001496+0000", 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4383999999999997e-05 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1266 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 950272~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213055576999999, 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014670829, 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1266 2024-09-17T21:47:39.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987313+0000", 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987314+0000", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987316+0000", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4059999999999999e-06 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987323+0000", 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2899999999999997e-06 2024-09-17T21:47:39.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987334+0000", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.093e-05 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001524+0000", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014189715 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001592+0000", 2024-09-17T21:47:39.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8132000000000004e-05 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.001984+0000", 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039185199999999999 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1267 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213026607000003, 2024-09-17T21:47:39.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014824201, 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1267 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987342+0000", 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987343+0000", 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987345+0000", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3429999999999998e-06 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987351+0000", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5339999999999996e-06 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987356+0000", 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7029999999999997e-06 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002014+0000", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014657411 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002084+0000", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0596e-05 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002166+0000", 2024-09-17T21:47:39.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2083999999999995e-05 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1268 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1605632~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.213004957999999, 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014992027999999999, 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.268 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1268 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987364+0000", 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6100000000000001e-07 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987369+0000", 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7129999999999996e-06 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987376+0000", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7039999999999997e-06 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987381+0000", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2390000000000002e-06 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002195+0000", 2024-09-17T21:47:39.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014814226 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002268+0000", 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3279000000000001e-05 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002356+0000", 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7305999999999994e-05 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.272 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5024.0:1269 3.1f 3:ffea25ac:::rbd_data.111297114b95.0000000000000000:19 [read 1933312~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 33.212977793999997, 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015291325999999999, 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5024", 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2628097216", 2024-09-17T21:47:39.273 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 1269 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987391+0000", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987394+0000", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3989999999999998e-06 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987401+0000", 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8210000000000004e-06 2024-09-17T21:47:39.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:05.987406+0000", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0030000000000002e-06 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002522+0000", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015116368 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002595+0000", 2024-09-17T21:47:39.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3335000000000002e-05 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.002682+0000", 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6927000000000003e-05 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:771 3.1 3:8006aab1:::rbd_data.1112f0bef54a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 8388608 write_size 8388608,write 8280576~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:39.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.596224741, 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013777717, 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 771 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:39.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604144+0000", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.604145+0000", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5199999999999996e-07 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608094+0000", 2024-09-17T21:47:39.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0039492950000000002 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608114+0000", 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0254e-05 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608122+0000", 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4229999999999999e-06 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608205+0000", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3511999999999996e-05 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608304+0000", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8524999999999996e-05 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.608621+0000", 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00031733999999999999 2024-09-17T21:47:39.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617430+0000", 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088086459999999998 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617787+0000", 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035710100000000002 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617854+0000", 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7699000000000004e-05 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:06.617922+0000", 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7169999999999996e-05 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:39.283 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.317527551000001, 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:39.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:39.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:39.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.310625779999999, 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:39.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:39.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.310509689, 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2165 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch ping cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.00427461, 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69677434100000002, 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2165 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196094+0000", 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196096+0000", 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196108+0000", 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2092999999999999e-05 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196130+0000", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1695000000000002e-05 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196140+0000", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7699999999999996e-06 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196216+0000", 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5899e-05 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.196227+0000", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1008e-05 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892718+0000", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.69649070000000002 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892759+0000", 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1876999999999999e-05 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892868+0000", 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010892599999999999 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.304 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:39.306 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:39.302+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:39.406 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:39.403+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:39.452 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:39.452 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:39.507 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:39.504+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:39.569 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.569 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:39.569 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:39.569 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:39.569 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:39.608 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:39.605+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.686 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779144106000004, 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:39.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:39.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:39.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779133805000001, 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779117730999999, 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779106518000006, 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779089080000006, 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.779078209999994, 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:39.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:39.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.778883332999996, 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:39.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:39.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.777178663000001, 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:39.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:39.722 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.777168024999995, 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.723 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:39.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.777157931999994, 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:39.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.777147812999999, 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.775017582999993, 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.775007852000002, 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:39.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:39.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:39.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:39.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.774224493000006, 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:39.745 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:39.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.774208923000003, 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.774116020999998, 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:39.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:39.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:39.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.773621402000003, 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:39.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.773477357000004, 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:39.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.772791920000003, 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.772429529999997, 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:39.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:39.775 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:39.777 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:39.775+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:39.878 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:39.876+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:39.925 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:39.925 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:39.926 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:39.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:39.980 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:47:39.982 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:39.980+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:40.028 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:40.060 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone35 from 0xec01c00 (overlap 0xec01c00) to 0x5e67000 2024-09-17T21:47:40.082 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:40.080+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:40.114 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:40.114 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:40.114 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:40.125 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:40.192 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:40.193 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:40.193 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:40.190+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:40.210 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:40.210 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:40.227 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:40.268 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:40.269 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:40.269 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:40.269 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:40.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:40.276 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:40.280 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:40.280 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:40.294 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:40.294 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:40.292+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:40.303 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:40.382 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:40.380+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:40.440 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:40.440 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:40.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:40.482 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:40.480+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:40.583 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:40.581+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:40.588 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:40.588 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:40.597 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:40.664 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:47:40.664 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:40.664 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:47:40.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:47:40.684 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:40.681+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:40.706 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:40.706 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:40.715 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:40.799 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:40.796+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:40.864 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:40.865 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:40.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:40.886 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:40.884+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:40.899 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-17T21:47:40.987 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:40.983+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:41.004 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 106, 2024-09-17T21:47:41.004 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:47:39.974776+0000", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:47:26.936709+0000", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:47:06.858395+0000", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:47:41.005 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:47:41.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:41.007 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:47:41.008 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:41.009 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:41.010 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:41.011 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.012 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 5.880000114440918, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 5.880000114440918, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.17000000178813934, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:41.014 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:41.015 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:41.016 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:41.017 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:41.018 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2699999809265137, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2699999809265137, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:41.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:41.020 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:41.021 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "106", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 38, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 106, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:41.022 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:41.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:41.024 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.5099999904632568, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.5099999904632568, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.25, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.25, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:41.025 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.026 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.029 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.032 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.033 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:41.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.037 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:47:41.042 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:41.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 97, 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:47:41.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/2260220328", 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/2260220328", 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/2260220328", 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/2260220328", 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:41.054 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:47:41.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:47:41.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:41.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.061 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 97 2024-09-17T21:47:41.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:41.063 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [ 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1b", 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: "mappings": [ 2024-09-17T21:47:41.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "from": 4, 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "to": 1 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:47:41.065 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:47:41.066 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:47:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:47:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:47:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:47:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:41.067 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:41.068 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.068 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.069 INFO:tasks.thrashosds.thrasher:j is {'epoch': 106, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:47:39.974776+0000', 'last_up_change': '2024-09-17T21:47:26.936709+0000', 'last_in_change': '2024-09-17T21:47:06.858395+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 38, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 0, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 81, 'down_at': 97, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 2260220328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 2260220328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 2260220328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 2260220328}]}, 'public_addr': '172.21.15.53:6817/2260220328', 'cluster_addr': '172.21.15.53:6819/2260220328', 'heartbeat_back_addr': '172.21.15.53:6823/2260220328', 'heartbeat_front_addr': '172.21.15.53:6821/2260220328', 'state': ['exists']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 97}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1b', 'mappings': [{'from': 4, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:47:41.069 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.1b 2024-09-17T21:47:41.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.1b 2024-09-17T21:47:41.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:41.109 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:41.107+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:41.188 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:41.185+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:41.288 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:41.286+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:41.348 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.348 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:41.401 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:41.399+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:41.465 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.465 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:41.515 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:41.513+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:41.549 INFO:teuthology.orchestra.run.smithi165.stdout:4247 punch from 0xca0c400 to 0xca19800, (0xd400 bytes) 2024-09-17T21:47:41.591 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:41.589+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:41.613 INFO:teuthology.orchestra.run.smithi165.stdout:4248 trunc from 0xec01c00 to 0x5512000 2024-09-17T21:47:41.626 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.626 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:41.721 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:41.719+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:41.794 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:41.791+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:41.811 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:41.811 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:41.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:41.845 INFO:teuthology.orchestra.run.smithi165.stdout:4249 punch from 0x2fa5200 to 0x2fb2c00, (0xda00 bytes) 2024-09-17T21:47:41.895 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:41.892+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:41.987 INFO:teuthology.orchestra.run.smithi007.stderr:clear 3.1b pg_upmap_items mapping 2024-09-17T21:47:41.995 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:41.993+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:41.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:47:42.023 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.023 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:42.116 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:42.114+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:42.183 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.183 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:42.237 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:42.235+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:42.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:42.298 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:42.295+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:42.359 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.359 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.370 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:42.398 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:42.396+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.478 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:42.530 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:42.528+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:42.595 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.596 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.600 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:42.598+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:42.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:42.701 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:42.699+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:42.712 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.713 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.721 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:42.802 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:42.800+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:42.827 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.828 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:42.903 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:42.900+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:42.943 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:42.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:47:43.035 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:43.033+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:43.098 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.098 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.104 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:43.102+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:43.107 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:43.205 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:43.203+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:43.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.254 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:43.306 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:43.304+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:43.369 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.369 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.379 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:43.419 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:43.417+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:43.503 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.503 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.508 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:43.506+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:43.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:43.610 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:43.607+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:43.622 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.622 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:43.711 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:43.709+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:43.738 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.738 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:43.812 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:43.810+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:43.857 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:43.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:43.950 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:43.948+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:44.013 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:44.011+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:44.032 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:44.032 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:44.038 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:44.038 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:44.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:44.074 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:44.075 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:44.105 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:44.106 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:44.106 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:44.110 INFO:teuthology.orchestra.run.smithi165.stdout:4252 read 0x1956e00 thru 0x1964598 (0xd799 bytes) 2024-09-17T21:47:44.113 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:44.111+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:44.135 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:44.135 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:44.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:44.166 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:44.170 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:44.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:44.214 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:44.212+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:44.226 INFO:teuthology.orchestra.run.smithi165.stdout:4253 trunc from 0x5512000 to 0x6ff7a00 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334808046000006, 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:44.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:44.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334797745000003, 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:44.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.253 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334781671000002, 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:44.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:44.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334770457999994, 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:44.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:44.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334753019999994, 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:44.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334742149999997, 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:44.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:44.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.334547272999998, 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.332842603000003, 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:44.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.332831964999997, 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:44.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:44.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:44.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:44.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:44.291 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.332821871999997, 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:44.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.332811753000001, 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.297 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.330681522999996, 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:44.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.330671792000004, 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:44.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:44.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:44.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.329888432999994, 2024-09-17T21:47:44.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:44.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.329872863000006, 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.329779961, 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.329285342000006, 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.329141297000007, 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.328455860000005, 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.328093469999999, 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.349 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:44.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:44.351 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:44.347+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:44.445 INFO:teuthology.orchestra.run.smithi165.stdout:4254 trunc from 0x6ff7a00 to 0x7f41c00 2024-09-17T21:47:44.450 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:44.448+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:44.464 INFO:teuthology.orchestra.run.smithi165.stdout:4255 read 0x5611800 thru 0x56121c9 (0x9ca bytes) 2024-09-17T21:47:44.473 INFO:teuthology.orchestra.run.smithi165.stdout:4256 punch from 0x6caa800 to 0x6cb4600, (0x9e00 bytes) 2024-09-17T21:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout:4257 write 0xe500800 thru 0xe50e607 (0xde08 bytes) 2024-09-17T21:47:44.518 INFO:teuthology.orchestra.run.smithi165.stdout:4266 read 0xaf59000 thru 0xaf5efcd (0x5fce bytes) 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi165.stdout:4269 read 0x249d200 thru 0x24a7a59 (0xa85a bytes) 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11891 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 303104~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503261+0000", 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013249284, 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013267058999999999, 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11891 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503261+0000", 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503261+0000", 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503261+0000", 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503262+0000", 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.342e-06 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503267+0000", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3919999999999996e-06 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503270+0000", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8260000000000001e-06 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.516428+0000", 2024-09-17T21:47:44.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013157975000000001 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.516510+0000", 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2107999999999994e-05 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11892 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503273+0000", 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013237349000000001, 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013296848999999999, 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11892 2024-09-17T21:47:44.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503273+0000", 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503273+0000", 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503273+0000", 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503274+0000", 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1850000000000001e-06 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503282+0000", 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7470000000000005e-06 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503287+0000", 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11893 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503291+0000", 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013219616, 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013333555, 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11893 2024-09-17T21:47:44.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503291+0000", 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503291+0000", 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503291+0000", 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3799999999999999e-07 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503292+0000", 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3349999999999999e-06 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503295+0000", 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2990000000000001e-06 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503298+0000", 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.537e-06 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11894 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503302+0000", 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013207966999999999, 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013376166, 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11894 2024-09-17T21:47:44.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503302+0000", 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503302+0000", 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503302+0000", 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.53e-07 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503304+0000", 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1960000000000001e-06 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503308+0000", 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0849999999999999e-06 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503310+0000", 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11895 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503313+0000", 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013196868, 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013402213, 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11895 2024-09-17T21:47:44.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503313+0000", 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503313+0000", 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503314+0000", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.23e-07 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503315+0000", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.387e-06 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503318+0000", 2024-09-17T21:47:44.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9919999999999999e-06 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503320+0000", 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4169999999999999e-06 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:11896 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e109)", 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:44.503323+0000", 2024-09-17T21:47:44.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013186956999999999, 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013420421, 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 11896 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503323+0000", 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503323+0000", 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503323+0000", 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503325+0000", 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0890000000000001e-06 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503327+0000", 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.959e-06 2024-09-17T21:47:44.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:44.503331+0000", 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6959999999999998e-06 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 6 2024-09-17T21:47:44.544 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout:4271 write 0xd189000 thru 0xd18ed08 (0x5d09 bytes) 2024-09-17T21:47:44.545 INFO:teuthology.orchestra.run.smithi165.stdout:4272 trunc from 0xe50e800 to 0xacfe600 2024-09-17T21:47:44.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:44.557 INFO:teuthology.orchestra.run.smithi165.stdout:4274 trunc from 0xacfe600 to 0x6df2c00 2024-09-17T21:47:44.589 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:44.587+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:44.596 INFO:teuthology.orchestra.run.smithi165.stdout:4275 punch from 0x50bc400 to 0x50c7000, (0xac00 bytes) 2024-09-17T21:47:44.603 INFO:teuthology.orchestra.run.smithi165.stdout:4276 read 0x1e34c00 thru 0x1e3d859 (0x8c5a bytes) 2024-09-17T21:47:44.605 INFO:teuthology.orchestra.run.smithi165.stdout:4277 clone 37 order 18 su 131072 sc 2 2024-09-17T21:47:44.652 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:44.650+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:44.673 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:44.673 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:44.673 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:44.673 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:44.673 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:44.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:44.753 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:44.751+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:44.831 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:44.831 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:44.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_repop(client.4673.0:711 3.d e50/21 3:b0b885dc:::rbd_data.1112abb63c65.0000000000000000:head v 50'267, mlcod=48'265)", 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.521968270000002, 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018924204999999999, 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306014+0000", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.306016+0000", 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:44.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313439+0000", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074228970000000003 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313462+0000", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3247999999999998e-05 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313472+0000", 2024-09-17T21:47:44.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0390999999999999e-05 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313544+0000", 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1224999999999997e-05 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.313570+0000", 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6270000000000001e-05 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324872+0000", 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011302614000000001 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.324938+0000", 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5776000000000002e-05 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.955209768000003, 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.837 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:44.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:44.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:44.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2273 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [zero 25690112~2621440] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.947687626000004, 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018409838000000001, 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2273 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880295+0000", 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880297+0000", 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.823e-06 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880300+0000", 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4199999999999999e-06 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880313+0000", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2852e-05 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880321+0000", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7619999999999995e-06 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888594+0000", 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082736070000000005 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.888667+0000", 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2677999999999998e-05 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898705+0000", 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010037696 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:16775 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.079564+0000", 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7484178310000003, 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018104273000000001, 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16775 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079564+0000", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079564+0000", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079565+0000", 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:44.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079572+0000", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.002e-06 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079579+0000", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9410000000000001e-06 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079584+0000", 2024-09-17T21:47:44.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1930000000000001e-06 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.095895+0000", 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016311196 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.096451+0000", 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055552499999999996 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097669+0000", 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012180120000000001 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:16776 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [watch reconnect cookie 140714604606752 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7483909710000001, 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021834859000000002, 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16776 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079592+0000", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079594+0000", 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-17T21:47:44.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079601+0000", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7139999999999996e-06 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079607+0000", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9839999999999998e-06 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.096582+0000", 2024-09-17T21:47:44.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016975463 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.096659+0000", 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7359000000000003e-05 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101426+0000", 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004766803 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:916 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify cookie 140500862586480] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.080617+0000", 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7473652269999995, 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017166055, 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 916 2024-09-17T21:47:44.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080617+0000", 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080617+0000", 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080618+0000", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6899999999999996e-07 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080623+0000", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.746e-06 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080632+0000", 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1200000000000008e-06 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080638+0000", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5740000000000001e-06 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097488+0000", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016849759999999998 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097558+0000", 2024-09-17T21:47:44.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0005000000000005e-05 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097783+0000", 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000225081 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:917 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify cookie 140500862644928] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.080650+0000", 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7473323169999997, 2024-09-17T21:47:44.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017358500999999998, 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 917 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080650+0000", 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080650+0000", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080651+0000", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3900000000000005e-07 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080654+0000", 2024-09-17T21:47:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0910000000000001e-06 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080661+0000", 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9369999999999998e-06 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.080666+0000", 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8000000000000004e-06 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097817+0000", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017150266000000001 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.097884+0000", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7791000000000002e-05 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.098009+0000", 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000124077 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:932 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.084124+0000", 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7438586699999998, 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016764504, 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 932 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084124+0000", 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084124+0000", 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084124+0000", 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000004e-07 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084127+0000", 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6790000000000001e-06 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084133+0000", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4409999999999998e-06 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084139+0000", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3600000000000004e-06 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.100263+0000", 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016124781000000001 2024-09-17T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.100794+0000", 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053010700000000004 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.100888+0000", 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4582999999999999e-05 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:933 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.084153+0000", 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7438297179999998, 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016917293999999999, 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 933 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084153+0000", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084153+0000", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084153+0000", 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7299999999999996e-07 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084156+0000", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7870000000000002e-06 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084163+0000", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2130000000000002e-06 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084168+0000", 2024-09-17T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.135e-06 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.100922+0000", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016754073000000001 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.100989+0000", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7024000000000001e-05 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101070+0000", 2024-09-17T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0488999999999995e-05 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:934 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.084176+0000", 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7438065949999997, 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017453338999999998, 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 934 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084176+0000", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084176+0000", 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084176+0000", 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084179+0000", 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.554e-06 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084186+0000", 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084191+0000", 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2519999999999999e-06 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101463+0000", 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017272189 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101539+0000", 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5607999999999998e-05 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101629+0000", 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0496999999999998e-05 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:935 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.084197+0000", 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7437849019999998, 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017808709999999998, 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 935 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084197+0000", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084197+0000", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084198+0000", 2024-09-17T21:47:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2200000000000004e-07 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084201+0000", 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5940000000000001e-06 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084207+0000", 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6359999999999997e-06 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084212+0000", 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1569999999999999e-06 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101661+0000", 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017448677999999999 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101738+0000", 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6699000000000005e-05 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102006+0000", 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026842400000000002 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:936 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.084219+0000", 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7437631429999998, 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017954597999999999, 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 936 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084219+0000", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084219+0000", 2024-09-17T21:47:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084220+0000", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084222+0000", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6809999999999998e-06 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084229+0000", 2024-09-17T21:47:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6669999999999999e-06 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.084234+0000", 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6e-06 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102037+0000", 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017802941999999999 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102100+0000", 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3305999999999993e-05 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102174+0000", 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3966000000000003e-05 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:939 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify cookie 140500862668464] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.085721+0000", 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.742261321, 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016659174999999998, 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 939 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085721+0000", 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085721+0000", 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085722+0000", 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6199999999999997e-07 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085727+0000", 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1869999999999996e-06 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085744+0000", 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7164e-05 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.085751+0000", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1849999999999998e-06 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102203+0000", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016451437999999999 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102264+0000", 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1295000000000002e-05 2024-09-17T21:47:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102380+0000", 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116144 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:951 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.086214+0000", 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.741767887, 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016740824000000001, 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 951 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086214+0000", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086214+0000", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086215+0000", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086218+0000", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9859999999999999e-06 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086225+0000", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.086235+0000", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0645000000000001e-05 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102802+0000", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016567056 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102864+0000", 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1690000000000001e-05 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102955+0000", 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1336999999999996e-05 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:960 3.18 3:18e70942:::rbd_data.1112ffdab28d.00000000000000ff:head [stat,delete] snapc 26=[26] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.088097+0000", 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7398854699999999, 2024-09-17T21:47:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016889781999999999, 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 960 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088097+0000", 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088097+0000", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088098+0000", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9600000000000008e-07 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088103+0000", 2024-09-17T21:47:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8069999999999999e-06 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088112+0000", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9400000000000008e-06 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088118+0000", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.601e-06 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.102991+0000", 2024-09-17T21:47:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014872445 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.103111+0000", 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000120829 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.104987+0000", 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018751639999999999 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:980 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify cookie 140500862522224] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.088259+0000", 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7397230740000005, 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016956697999999999, 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 980 2024-09-17T21:47:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088259+0000", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088259+0000", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088260+0000", 2024-09-17T21:47:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088262+0000", 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6139999999999999e-06 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088269+0000", 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3960000000000003e-06 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.088274+0000", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0340000000000004e-06 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105047+0000", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016772777999999999 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105092+0000", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5157999999999997e-05 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105216+0000", 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012426700000000001 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:981 3.8 3:11a2edfd:::rbd_data.1112ffdab28d.00000000000000cc:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 26=[26] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.089713+0000", 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7382691770000003, 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017409213999999999, 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 981 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089713+0000", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089713+0000", 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089714+0000", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017e-06 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089720+0000", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4870000000000002e-06 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089731+0000", 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:47:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089738+0000", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3250000000000002e-06 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105031+0000", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015292534999999999 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105098+0000", 2024-09-17T21:47:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6633999999999999e-05 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105297+0000", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000199717 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107032+0000", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017344089999999999 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107043+0000", 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1156e-05 2024-09-17T21:47:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107087+0000", 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3804000000000003e-05 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107122+0000", 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5754999999999999e-05 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:983 3.8 3:17924b86:::rbd_data.1112ffdab28d.00000000000000e1:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 26=[26] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.089789+0000", 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7381932100000004, 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017862780000000002, 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 983 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089789+0000", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089789+0000", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089790+0000", 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3900000000000005e-07 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089793+0000", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4860000000000002e-06 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089922+0000", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000128899 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089930+0000", 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7740000000000004e-06 2024-09-17T21:47:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106127+0000", 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016196981999999999 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106202+0000", 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5339000000000005e-05 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 1", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106389+0000", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018650399999999999 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107344+0000", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00095572099999999998 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107567+0000", 2024-09-17T21:47:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022252099999999999 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107617+0000", 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999999e-05 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107652+0000", 2024-09-17T21:47:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5165e-05 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:984 3.18 3:1afd3ef9:::rbd_data.1112ffdab28d.00000000000000e0:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc 26=[26] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.089943+0000", 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7380392899999997, 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017468706000000001, 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 984 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089943+0000", 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089943+0000", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089944+0000", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.92e-07 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089948+0000", 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.155e-06 2024-09-17T21:47:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089956+0000", 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.977e-06 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089962+0000", 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6459999999999998e-06 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105724+0000", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015762938000000001 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.105915+0000", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000190729 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 0", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106099+0000", 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018409500000000001 2024-09-17T21:47:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107301+0000", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0012021180000000001 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107348+0000", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6387999999999997e-05 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107392+0000", 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4632999999999998e-05 2024-09-17T21:47:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.107412+0000", 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9335e-05 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5620.0:990 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [notify cookie 140500862534144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.089975+0000", 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.738007638, 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016725976, 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5620", 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/4018580005", 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 990 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089975+0000", 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089975+0000", 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089975+0000", 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089978+0000", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2289999999999999e-06 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089986+0000", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.089992+0000", 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8329999999999999e-06 2024-09-17T21:47:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106498+0000", 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016505944000000002 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106567+0000", 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9561999999999996e-05 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.106701+0000", 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000133442 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:44.929 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:44.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:44.931 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:44.928+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:45.007 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:47:45.045 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:45.045 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:45.045 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.051 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:45.049+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:45.055 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:45.075 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone36 from 0x6df2c00 (overlap 0x5512000) to 0x440e600 2024-09-17T21:47:45.131 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:45.129+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:45.210 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.210 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:45.211 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:45.211 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:45.211 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.219 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:45.263 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:45.261+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.371422897000002, 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:45.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.357436840999995, 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:45.333 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:45.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:45.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.355475634000001, 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:45.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.353442276999999, 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:45.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.351464359000005, 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:45.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.351441565000002, 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:45.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:45.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.345532501999998, 2024-09-17T21:47:45.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:45.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.343501442999994, 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:45.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:45.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:45.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.343474869000005, 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:45.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:45.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.342865075999995, 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:45.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:45.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.342847489999997, 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.378 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:45.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:45.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.342832533000006, 2024-09-17T21:47:45.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:45.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:45.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.627151045000005, 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:45.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:45.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.627120423999997, 2024-09-17T21:47:45.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:45.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:45.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.627095806, 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:45.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:45.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:45.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.627073859000006, 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:45.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:45.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.627052388999999, 2024-09-17T21:47:45.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:45.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:45.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:45.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.585635585999995, 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:45.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:45.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.584591126000007, 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:45.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:45.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.383405795000002, 2024-09-17T21:47:45.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:45.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:45.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:45.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.440 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.441 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:45.442 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:45.439+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:45.542 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:45.540+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:45.589 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.589 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:45.589 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:45.589 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.598 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:45.643 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:45.641+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:45.704 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.704 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:45.705 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:45.705 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:45.757 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:45.755+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:45.822 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:45.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.848056439999993, 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:45.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:45.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.848033947000005, 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:45.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.848009687000001, 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:45.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:45.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847983803999995, 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:45.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:45.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847958802999997, 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:45.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:45.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847919004000005, 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:45.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:45.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847903259000006, 2024-09-17T21:47:45.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:45.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:45.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847891723999993, 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.858 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.847881423999993, 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:45.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:45.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845910180999994, 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:45.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845897033, 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:45.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:45.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845886426999996, 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845876089000001, 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:45.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845866082000001, 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:45.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845856131000005, 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:45.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:45.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.845840823000003, 2024-09-17T21:47:45.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:45.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:45.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.843864936000003, 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.895 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:45.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:45.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.843853980000006, 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:45.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:45.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:45.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.843841491000006, 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 88.843831027999997, 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:45.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:45.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:45.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:45.913 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:45.915 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:45.912+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:45.915 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:45.997 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:47:46.015 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:46.013+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:46.062 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:46.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14568 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028375+0000", 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013217543999999999, 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013242365000000001, 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14568 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028375+0000", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028375+0000", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028375+0000", 2024-09-17T21:47:46.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028377+0000", 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.889e-06 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028384+0000", 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5320000000000003e-06 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028389+0000", 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1429999999999997e-06 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14736 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.029997+0000", 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011595196, 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011685413, 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14736 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.029997+0000", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.029997+0000", 2024-09-17T21:47:46.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.029998+0000", 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.44e-07 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030000+0000", 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2280000000000001e-06 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030008+0000", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9379999999999992e-06 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030013+0000", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1170000000000003e-06 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14816 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030805+0000", 2024-09-17T21:47:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010786924999999999, 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010943881000000001, 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14816 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030805+0000", 2024-09-17T21:47:46.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030805+0000", 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030806+0000", 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030808+0000", 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030814+0000", 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3409999999999999e-06 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030826+0000", 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1928e-05 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14900 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031643+0000", 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0099491379999999997, 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010161455, 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14900 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031643+0000", 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031643+0000", 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031644+0000", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.82e-07 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031650+0000", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3300000000000004e-06 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031657+0000", 2024-09-17T21:47:46.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1579999999999999e-06 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031662+0000", 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1510000000000003e-06 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15132 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032534+0000", 2024-09-17T21:47:46.076 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0090585790000000006, 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0093263579999999999, 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15132 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032534+0000", 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032534+0000", 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032534+0000", 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032536+0000", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1610000000000001e-06 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032543+0000", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8759999999999999e-06 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032548+0000", 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1159999999999998e-06 2024-09-17T21:47:46.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15213 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034004+0000", 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0075884409999999996, 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079155760000000006, 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15213 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034004+0000", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034004+0000", 2024-09-17T21:47:46.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034004+0000", 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8699999999999995e-07 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034007+0000", 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3640000000000002e-06 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034014+0000", 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034019+0000", 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0549999999999999e-06 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15290 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037094+0000", 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0044978930000000002, 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048804349999999998, 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15290 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037094+0000", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037094+0000", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037095+0000", 2024-09-17T21:47:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037106+0000", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1265e-05 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037115+0000", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.952e-06 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037121+0000", 2024-09-17T21:47:46.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8289999999999996e-06 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15374 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039742+0000", 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0018504350000000001, 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022880869999999998, 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15374 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039742+0000", 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039742+0000", 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039742+0000", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039744+0000", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7689999999999999e-06 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039751+0000", 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9990000000000002e-06 2024-09-17T21:47:46.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039760+0000", 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8210000000000008e-06 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14277 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.026850+0000", 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.014742463000000001, 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015236639999999999, 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14277 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026850+0000", 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026850+0000", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026850+0000", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4000000000000002e-07 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026852+0000", 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8729999999999999e-06 2024-09-17T21:47:46.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026858+0000", 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026864+0000", 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2100000000000001e-06 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14825 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030833+0000", 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.01075899, 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011313261999999999, 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14825 2024-09-17T21:47:46.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030833+0000", 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030833+0000", 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030834+0000", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.44e-07 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030836+0000", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030842+0000", 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:47:46.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030847+0000", 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7530000000000001e-06 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14905 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031669+0000", 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0099229079999999994, 2024-09-17T21:47:46.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010534389, 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14905 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031669+0000", 2024-09-17T21:47:46.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031669+0000", 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031670+0000", 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.96e-07 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031672+0000", 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031678+0000", 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2570000000000004e-06 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031683+0000", 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15137 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032562+0000", 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0090300450000000004, 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096966299999999995, 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15137 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032562+0000", 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032562+0000", 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032563+0000", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8299999999999997e-07 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032565+0000", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3089999999999998e-06 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032572+0000", 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0190000000000001e-06 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032578+0000", 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9440000000000002e-06 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15217 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034026+0000", 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0075663960000000004, 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082941939999999995, 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15217 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034026+0000", 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034026+0000", 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034026+0000", 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6300000000000005e-07 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034029+0000", 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034036+0000", 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7179999999999999e-06 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034041+0000", 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1100000000000002e-06 2024-09-17T21:47:46.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15298 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037130+0000", 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0044628030000000004, 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052494070000000002, 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.108 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15298 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037130+0000", 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037130+0000", 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037130+0000", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037132+0000", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3190000000000002e-06 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037139+0000", 2024-09-17T21:47:46.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6900000000000003e-06 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037144+0000", 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3310000000000003e-06 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15383 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039766+0000", 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001826182, 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026680139999999998, 2024-09-17T21:47:46.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15383 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039766+0000", 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039766+0000", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039766+0000", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039775+0000", 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9970000000000004e-06 2024-09-17T21:47:46.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039783+0000", 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3050000000000004e-06 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039788+0000", 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0479999999999998e-06 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14278 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.026870+0000", 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.014722673, 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015623249000000001, 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14278 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026870+0000", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026870+0000", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026870+0000", 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:46.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026876+0000", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2949999999999999e-06 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026883+0000", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4470000000000003e-06 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026888+0000", 2024-09-17T21:47:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1000000000000003e-06 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14826 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030853+0000", 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010738937, 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011698053999999999, 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14826 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030853+0000", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030853+0000", 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030854+0000", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030856+0000", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030862+0000", 2024-09-17T21:47:46.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4960000000000001e-06 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030868+0000", 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14906 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031691+0000", 2024-09-17T21:47:46.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0099009619999999993, 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010926788999999999, 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14906 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031691+0000", 2024-09-17T21:47:46.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031691+0000", 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031692+0000", 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031701+0000", 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031709+0000", 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6320000000000008e-06 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031714+0000", 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3859999999999998e-06 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15138 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032595+0000", 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0089968229999999993, 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010087822, 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15138 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032595+0000", 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032595+0000", 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032596+0000", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8800000000000002e-07 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032598+0000", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032607+0000", 2024-09-17T21:47:46.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8380000000000008e-06 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032612+0000", 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2519999999999999e-06 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15218 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034047+0000", 2024-09-17T21:47:46.128 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0075452840000000002, 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0086987520000000006, 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15218 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034047+0000", 2024-09-17T21:47:46.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034047+0000", 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034047+0000", 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034050+0000", 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.435e-06 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034056+0000", 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4309999999999999e-06 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034068+0000", 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1902999999999999e-05 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15299 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037151+0000", 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0044412699999999998, 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056523379999999998, 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15299 2024-09-17T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037151+0000", 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037151+0000", 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037151+0000", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.84e-07 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037153+0000", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9970000000000001e-06 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037160+0000", 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1229999999999997e-06 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037165+0000", 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.006e-06 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15384 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039796+0000", 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001796707, 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0030691389999999998, 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15384 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039796+0000", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039796+0000", 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039796+0000", 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039798+0000", 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.367e-06 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039805+0000", 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4829999999999996e-06 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039816+0000", 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1269e-05 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14279 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.026904+0000", 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.014688789000000001, 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0160199, 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14279 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026904+0000", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026904+0000", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026904+0000", 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4000000000000002e-07 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026906+0000", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.063e-06 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026912+0000", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026917+0000", 2024-09-17T21:47:46.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14827 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030874+0000", 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010718037999999999, 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012107517999999999, 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14827 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030874+0000", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030874+0000", 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030875+0000", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7300000000000001e-07 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030877+0000", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.888e-06 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030883+0000", 2024-09-17T21:47:46.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1360000000000002e-06 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030887+0000", 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7310000000000001e-06 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14907 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031728+0000", 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0098647319999999993, 2024-09-17T21:47:46.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011314571000000001, 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14907 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031728+0000", 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031728+0000", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031728+0000", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-07 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031730+0000", 2024-09-17T21:47:46.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.232e-06 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031737+0000", 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031742+0000", 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7929999999999997e-06 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15139 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032620+0000", 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0089727350000000008, 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010484457000000001, 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15139 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.149 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032620+0000", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032620+0000", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032620+0000", 2024-09-17T21:47:46.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7500000000000001e-07 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032622+0000", 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8479999999999999e-06 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032628+0000", 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.246e-06 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032633+0000", 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4909999999999999e-06 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15219 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034076+0000", 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.007516717, 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0090866000000000002, 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15219 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034076+0000", 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034076+0000", 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034076+0000", 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034078+0000", 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3e-06 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034085+0000", 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7569999999999998e-06 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034098+0000", 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2486999999999999e-05 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15300 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037171+0000", 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0044213969999999997, 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060493129999999997, 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15300 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037171+0000", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037171+0000", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037171+0000", 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:46.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037173+0000", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0499999999999999e-06 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037180+0000", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037185+0000", 2024-09-17T21:47:46.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9409999999999998e-06 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15385 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039825+0000", 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001766901, 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0034534090000000002, 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15385 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039825+0000", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039825+0000", 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039826+0000", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039833+0000", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039840+0000", 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:47:46.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039878+0000", 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7592000000000002e-05 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14280 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.026923+0000", 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.014669619, 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016422273000000001, 2024-09-17T21:47:46.162 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14280 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026923+0000", 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026923+0000", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026923+0000", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026925+0000", 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8029999999999999e-06 2024-09-17T21:47:46.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026931+0000", 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6170000000000003e-06 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.026936+0000", 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5390000000000001e-06 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14828 3.1f 3:fb4f96ca:::rbd_data.11126af432c4.000000000000000e:25 [read 3608576~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030893+0000", 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010699581999999999, 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012519753999999999, 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14828 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030893+0000", 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030893+0000", 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030893+0000", 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5400000000000002e-07 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030895+0000", 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6819999999999999e-06 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030901+0000", 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0859999999999998e-06 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030905+0000", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5909999999999998e-06 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043241+0000", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012335656 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043312+0000", 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1316999999999995e-05 2024-09-17T21:47:46.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043414+0000", 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101981 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14909 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031747+0000", 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0098449399999999999, 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011745401000000001, 2024-09-17T21:47:46.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14909 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031747+0000", 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031747+0000", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031748+0000", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031750+0000", 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.778e-06 2024-09-17T21:47:46.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031755+0000", 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8540000000000002e-06 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031769+0000", 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3397999999999999e-05 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15140 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032638+0000", 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0089540720000000004, 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010923514, 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15140 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032638+0000", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032638+0000", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032639+0000", 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100000000000001e-07 2024-09-17T21:47:46.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032644+0000", 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6250000000000004e-06 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032650+0000", 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1550000000000004e-06 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032655+0000", 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6609999999999999e-06 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15220 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034105+0000", 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0074874039999999996, 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095167080000000005, 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15220 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034105+0000", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034105+0000", 2024-09-17T21:47:46.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034105+0000", 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6300000000000005e-07 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034108+0000", 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.26e-06 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034114+0000", 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6669999999999999e-06 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034119+0000", 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8300000000000003e-06 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15301 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037191+0000", 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0044014479999999996, 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065373380000000002, 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15301 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037191+0000", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037191+0000", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037191+0000", 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7e-07 2024-09-17T21:47:46.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037193+0000", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.883e-06 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037199+0000", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3570000000000003e-06 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037204+0000", 2024-09-17T21:47:46.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8509999999999998e-06 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15386 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039888+0000", 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001704562, 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.003897471, 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15386 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039888+0000", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039888+0000", 2024-09-17T21:47:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039888+0000", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1900000000000002e-07 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039891+0000", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7609999999999999e-06 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039908+0000", 2024-09-17T21:47:46.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6520000000000001e-05 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039913+0000", 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7359999999999998e-06 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14585 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028495+0000", 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013097114, 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015359581000000001, 2024-09-17T21:47:46.187 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14585 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028495+0000", 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028495+0000", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028496+0000", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028498+0000", 2024-09-17T21:47:46.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3e-06 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028505+0000", 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0199999999999997e-06 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028510+0000", 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1420000000000001e-06 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14753 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030114+0000", 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011478767000000001, 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013799906000000001, 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14753 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030114+0000", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030114+0000", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030114+0000", 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030126+0000", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1565e-05 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030133+0000", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.559e-06 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030144+0000", 2024-09-17T21:47:46.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1012000000000001e-05 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14837 3.17 3:edd050a8:::rbd_data.11126af432c4.0000000000000009:25 [read 3729408~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030911+0000", 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010681687, 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013068732, 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14837 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030911+0000", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030911+0000", 2024-09-17T21:47:46.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030911+0000", 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030913+0000", 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7400000000000001e-06 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030919+0000", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8069999999999996e-06 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030924+0000", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3140000000000003e-06 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043879+0000", 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012955253999999999 2024-09-17T21:47:46.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043959+0000", 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0284000000000006e-05 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14917 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031776+0000", 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0098163869999999993, 2024-09-17T21:47:46.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012273210999999999, 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14917 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031776+0000", 2024-09-17T21:47:46.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031776+0000", 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031776+0000", 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031779+0000", 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2089999999999999e-06 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031785+0000", 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8970000000000002e-06 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031791+0000", 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15149 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032662+0000", 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.008930782, 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011450147000000001, 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15149 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032662+0000", 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032662+0000", 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032662+0000", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5899999999999996e-07 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032664+0000", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9290000000000001e-06 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032671+0000", 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5180000000000001e-06 2024-09-17T21:47:46.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032675+0000", 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7990000000000001e-06 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15229 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034125+0000", 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0074672699999999998, 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010044557000000001, 2024-09-17T21:47:46.205 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15229 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034125+0000", 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034125+0000", 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034125+0000", 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5499999999999999e-07 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034127+0000", 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.77e-06 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034133+0000", 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0129999999999999e-06 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034138+0000", 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5650000000000003e-06 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15306 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037211+0000", 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0043813480000000002, 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0070198819999999999, 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15306 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037211+0000", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037211+0000", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037211+0000", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9999999999999998e-07 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037213+0000", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9800000000000001e-06 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037220+0000", 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4910000000000002e-06 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037225+0000", 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9570000000000001e-06 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15391 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039921+0000", 2024-09-17T21:47:46.212 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001671533, 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0043671339999999999, 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15391 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039921+0000", 2024-09-17T21:47:46.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039921+0000", 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039921+0000", 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039923+0000", 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0839999999999999e-06 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039930+0000", 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4989999999999999e-06 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039935+0000", 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.741e-06 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14586 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028518+0000", 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013074563000000001, 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015835175, 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14586 2024-09-17T21:47:46.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028518+0000", 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028518+0000", 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028518+0000", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.15e-07 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028520+0000", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0150000000000002e-06 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028527+0000", 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6259999999999997e-06 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028538+0000", 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869e-05 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14754 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030154+0000", 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011438705, 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014257168000000001, 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14754 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030154+0000", 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030154+0000", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030154+0000", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-07 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030161+0000", 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7410000000000003e-06 2024-09-17T21:47:46.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030168+0000", 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0280000000000003e-06 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030175+0000", 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6089999999999997e-06 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14838 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030930+0000", 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010662522000000001, 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013545391, 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14838 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030930+0000", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030930+0000", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030930+0000", 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0500000000000004e-07 2024-09-17T21:47:46.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030932+0000", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7090000000000001e-06 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030938+0000", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.66e-06 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030943+0000", 2024-09-17T21:47:46.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8520000000000003e-06 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14918 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031798+0000", 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0097947480000000007, 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012737425, 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14918 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031798+0000", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031798+0000", 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031798+0000", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9299999999999998e-07 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031800+0000", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2500000000000001e-06 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031807+0000", 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:47:46.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031812+0000", 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9640000000000002e-06 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15150 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032682+0000", 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0089101110000000001, 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011910567, 2024-09-17T21:47:46.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15150 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032682+0000", 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032682+0000", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032683+0000", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8599999999999998e-07 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032685+0000", 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1320000000000001e-06 2024-09-17T21:47:46.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032692+0000", 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.81e-06 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032705+0000", 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3740000000000001e-05 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15230 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034143+0000", 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0074489830000000002, 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010512084999999999, 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15230 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.233 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034143+0000", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034143+0000", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034144+0000", 2024-09-17T21:47:46.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034145+0000", 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.68e-06 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034151+0000", 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9549999999999997e-06 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034156+0000", 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.493e-06 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15307 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037231+0000", 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0043615370000000004, 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074864859999999997, 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15307 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037231+0000", 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037231+0000", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037231+0000", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037233+0000", 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.902e-06 2024-09-17T21:47:46.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037239+0000", 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037250+0000", 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1049999999999999e-05 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15392 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039940+0000", 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0016519939999999999, 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048392929999999997, 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15392 2024-09-17T21:47:46.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039940+0000", 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039940+0000", 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039941+0000", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.89e-07 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039943+0000", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7990000000000001e-06 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039948+0000", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8769999999999998e-06 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039953+0000", 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6670000000000004e-06 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14587 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028545+0000", 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013047497999999999, 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016294795000000001, 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.243 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14587 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028545+0000", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028545+0000", 2024-09-17T21:47:46.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028545+0000", 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4199999999999996e-07 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028547+0000", 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0600000000000002e-06 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028554+0000", 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7490000000000001e-06 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028560+0000", 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3739999999999997e-06 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14755 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030181+0000", 2024-09-17T21:47:46.246 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011411487, 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014720703999999999, 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14755 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030181+0000", 2024-09-17T21:47:46.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030181+0000", 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030181+0000", 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6600000000000002e-07 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030183+0000", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.801e-06 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030189+0000", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1229999999999997e-06 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030194+0000", 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:47:46.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14839 3.17 3:edd050a8:::rbd_data.11126af432c4.0000000000000009:25 [read 4046848~2048] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030950+0000", 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010641952999999999, 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014010918000000001, 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.250 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14839 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030950+0000", 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030950+0000", 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030951+0000", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8599999999999998e-07 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030958+0000", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3889999999999999e-06 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030966+0000", 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8990000000000001e-06 2024-09-17T21:47:46.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030972+0000", 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9399999999999999e-06 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.044892+0000", 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013920125 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:46.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.044959+0000", 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6743999999999997e-05 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14919 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031835+0000", 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0097573150000000008, 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013196714, 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14919 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031835+0000", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031835+0000", 2024-09-17T21:47:46.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031836+0000", 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031838+0000", 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4540000000000001e-06 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031845+0000", 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2910000000000002e-06 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031851+0000", 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1900000000000003e-06 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15151 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032714+0000", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0088783009999999999, 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012374995999999999, 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15151 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032714+0000", 2024-09-17T21:47:46.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032714+0000", 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032714+0000", 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032717+0000", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2110000000000001e-06 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032723+0000", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6969999999999996e-06 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032729+0000", 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3129999999999998e-06 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.261 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15231 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034161+0000", 2024-09-17T21:47:46.263 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.007431071, 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010988441, 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15231 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034161+0000", 2024-09-17T21:47:46.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034161+0000", 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034162+0000", 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034163+0000", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.778e-06 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034169+0000", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.959e-06 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034174+0000", 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3939999999999998e-06 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15308 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037258+0000", 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0043347689999999996, 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079473779999999997, 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15308 2024-09-17T21:47:46.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037258+0000", 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037258+0000", 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037258+0000", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2e-07 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037260+0000", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2189999999999998e-06 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037267+0000", 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.939e-06 2024-09-17T21:47:46.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037278+0000", 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0893999999999999e-05 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15393 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039958+0000", 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0016338170000000001, 2024-09-17T21:47:46.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0053071300000000002, 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15393 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039958+0000", 2024-09-17T21:47:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039958+0000", 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039959+0000", 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039961+0000", 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8589999999999999e-06 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039967+0000", 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0499999999999997e-06 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039972+0000", 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1410000000000004e-06 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14588 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028567+0000", 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.013025524, 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016759455999999999, 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14588 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028567+0000", 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028567+0000", 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028567+0000", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1899999999999998e-07 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028569+0000", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2170000000000001e-06 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028576+0000", 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5309999999999998e-06 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028581+0000", 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.758e-06 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14756 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030200+0000", 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011392352, 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015185888999999999, 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.277 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14756 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030200+0000", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030200+0000", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030200+0000", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030202+0000", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.863e-06 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030210+0000", 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5190000000000003e-06 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030215+0000", 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-06 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14920 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031857+0000", 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.009734889, 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013585664000000001, 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14920 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031857+0000", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031857+0000", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031858+0000", 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.89e-07 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031867+0000", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7090000000000008e-06 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031874+0000", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1919999999999999e-06 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031879+0000", 2024-09-17T21:47:46.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2639999999999999e-06 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15152 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032735+0000", 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0088569679999999998, 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012767506, 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15152 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032735+0000", 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032735+0000", 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032736+0000", 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9299999999999999e-07 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032738+0000", 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032745+0000", 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7800000000000003e-06 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032757+0000", 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.222e-05 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15232 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.287 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034179+0000", 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.007413017, 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011377672, 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15232 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034179+0000", 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034179+0000", 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034180+0000", 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034187+0000", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034194+0000", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.776e-06 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034199+0000", 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0749999999999997e-06 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15309 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037286+0000", 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.004306682, 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0083274979999999992, 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15309 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037286+0000", 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037286+0000", 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037286+0000", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3300000000000002e-07 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037289+0000", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3530000000000002e-06 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037296+0000", 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.486e-06 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037301+0000", 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15394 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039987+0000", 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0016049040000000001, 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056795509999999997, 2024-09-17T21:47:46.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15394 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039987+0000", 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039987+0000", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039988+0000", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9999999999999997e-07 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039991+0000", 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.57e-06 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039998+0000", 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0899999999999999e-06 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040003+0000", 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3260000000000003e-06 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14845 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031035+0000", 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010557798, 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014788918999999999, 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14845 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031035+0000", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031035+0000", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031035+0000", 2024-09-17T21:47:46.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031037+0000", 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2979999999999999e-06 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031044+0000", 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0439999999999998e-06 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031054+0000", 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8870000000000004e-06 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14925 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031887+0000", 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0097053060000000003, 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013994334000000001, 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14925 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031887+0000", 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031887+0000", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031887+0000", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-07 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031890+0000", 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.215e-06 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031896+0000", 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8050000000000001e-06 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031901+0000", 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9189999999999998e-06 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15157 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032783+0000", 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.008808962, 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013151461, 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15157 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032783+0000", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032783+0000", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032784+0000", 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9500000000000002e-07 2024-09-17T21:47:46.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032794+0000", 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0180000000000001e-05 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032803+0000", 2024-09-17T21:47:46.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6090000000000001e-06 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032809+0000", 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.584e-06 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15240 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034206+0000", 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.007386536, 2024-09-17T21:47:46.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011787532, 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15240 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034206+0000", 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034206+0000", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034206+0000", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034208+0000", 2024-09-17T21:47:46.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9750000000000001e-06 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034214+0000", 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1739999999999997e-06 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034219+0000", 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8210000000000001e-06 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15317 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037308+0000", 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0042845299999999999, 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0087396460000000002, 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15317 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037308+0000", 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037308+0000", 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037308+0000", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037310+0000", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1399999999999998e-06 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037317+0000", 2024-09-17T21:47:46.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.55e-06 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037322+0000", 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9799999999999998e-06 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15403 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.040010+0000", 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0015822259999999999, 2024-09-17T21:47:46.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060965940000000003, 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15403 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040010+0000", 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040010+0000", 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040011+0000", 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9800000000000004e-07 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040013+0000", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9889999999999999e-06 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040019+0000", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040024+0000", 2024-09-17T21:47:46.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14846 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031061+0000", 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010531512999999999, 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01510363, 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14846 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031061+0000", 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031061+0000", 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031061+0000", 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031063+0000", 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.933e-06 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031070+0000", 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031074+0000", 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6439999999999999e-06 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14926 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031908+0000", 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0096845509999999996, 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014316176999999999, 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14926 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031908+0000", 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031908+0000", 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031908+0000", 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-07 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031910+0000", 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0310000000000001e-06 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031917+0000", 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.443e-06 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031922+0000", 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8949999999999997e-06 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15158 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032818+0000", 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0087745029999999995, 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013463018, 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15158 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032818+0000", 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032818+0000", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032818+0000", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9800000000000003e-07 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032821+0000", 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2089999999999999e-06 2024-09-17T21:47:46.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032827+0000", 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8000000000000001e-06 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032832+0000", 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0730000000000004e-06 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15241 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034225+0000", 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0073668960000000004, 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012117031, 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15241 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034225+0000", 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034225+0000", 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034705+0000", 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047957300000000001 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034717+0000", 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1533e-05 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034732+0000", 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5719999999999999e-05 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034740+0000", 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8660000000000006e-06 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15318 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037329+0000", 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0042635390000000002, 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.009066312, 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15318 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037329+0000", 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037329+0000", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037329+0000", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-07 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037331+0000", 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0449999999999999e-06 2024-09-17T21:47:46.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037338+0000", 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3500000000000002e-06 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037343+0000", 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9350000000000002e-06 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15404 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.040030+0000", 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001561998, 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064100720000000002, 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15404 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040030+0000", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.040030+0000", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042632+0000", 2024-09-17T21:47:46.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0026020420000000002 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042637+0000", 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3730000000000003e-06 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042648+0000", 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1192999999999999e-05 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042663+0000", 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5495999999999999e-05 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14847 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031079+0000", 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010512861, 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015421884, 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14847 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031079+0000", 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031079+0000", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031080+0000", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.53e-07 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031081+0000", 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.654e-06 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031087+0000", 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8309999999999997e-06 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031092+0000", 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5809999999999998e-06 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14927 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031928+0000", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0096642900000000007, 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014634291000000001, 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14927 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031928+0000", 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031928+0000", 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031929+0000", 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4000000000000002e-07 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031931+0000", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0949999999999998e-06 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031937+0000", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031942+0000", 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15159 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032839+0000", 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0087530820000000006, 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013780570000000001, 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15159 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032839+0000", 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032839+0000", 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032840+0000", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032842+0000", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0059999999999999e-06 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032848+0000", 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4609999999999996e-06 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032853+0000", 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7879999999999997e-06 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15242 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034752+0000", 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0068399710000000002, 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011929073, 2024-09-17T21:47:46.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15242 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034752+0000", 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034752+0000", 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034753+0000", 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5499999999999998e-07 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034756+0000", 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6819999999999999e-06 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034763+0000", 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6380000000000004e-06 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034769+0000", 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7130000000000002e-06 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15319 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037349+0000", 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.004243152, 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0093921000000000004, 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15319 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037349+0000", 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037349+0000", 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037415+0000", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5650999999999999e-05 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037428+0000", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3549e-05 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037438+0000", 2024-09-17T21:47:46.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8430000000000005e-06 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037444+0000", 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2110000000000003e-06 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15405 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042676+0000", 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9989161, 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0041274190000000002, 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15405 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042676+0000", 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042676+0000", 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042677+0000", 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-07 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042680+0000", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8480000000000001e-06 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042687+0000", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2629999999999997e-06 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042705+0000", 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7617000000000001e-05 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14848 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031097+0000", 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010495111999999999, 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015763810999999999, 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.361 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14848 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031097+0000", 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031097+0000", 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031098+0000", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1699999999999998e-07 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031099+0000", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5600000000000001e-06 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031105+0000", 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9859999999999999e-06 2024-09-17T21:47:46.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031110+0000", 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5569999999999997e-06 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14928 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031949+0000", 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0096435240000000005, 2024-09-17T21:47:46.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014987823000000001, 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14928 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031949+0000", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031949+0000", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031949+0000", 2024-09-17T21:47:46.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-07 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031961+0000", 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1579e-05 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031968+0000", 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6529999999999994e-06 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031983+0000", 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4277000000000001e-05 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15160 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032859+0000", 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0087330169999999992, 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014146565999999999, 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15160 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032859+0000", 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032859+0000", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032860+0000", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032862+0000", 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.968e-06 2024-09-17T21:47:46.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032869+0000", 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7819999999999996e-06 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032874+0000", 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1290000000000004e-06 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15243 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034777+0000", 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0068152630000000002, 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012286639, 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15243 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034777+0000", 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034777+0000", 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034778+0000", 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8999999999999997e-07 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034780+0000", 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3029999999999998e-06 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034787+0000", 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0870000000000001e-06 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034799+0000", 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2249000000000001e-05 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15326 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037455+0000", 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0041375839999999997, 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0096790580000000008, 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15326 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037455+0000", 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037455+0000", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037455+0000", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-07 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037458+0000", 2024-09-17T21:47:46.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5229999999999998e-06 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037466+0000", 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0590000000000003e-06 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037471+0000", 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15406 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042714+0000", 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.998878, 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044814069999999998, 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15406 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042714+0000", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042714+0000", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042715+0000", 2024-09-17T21:47:46.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042725+0000", 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0098999999999999e-05 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042733+0000", 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9289999999999998e-06 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042738+0000", 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5010000000000004e-06 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14605 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028677+0000", 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.012915639, 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018585983, 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14605 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028677+0000", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028677+0000", 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028677+0000", 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2499999999999995e-07 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028679+0000", 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9690000000000001e-06 2024-09-17T21:47:46.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028686+0000", 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3550000000000001e-06 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028690+0000", 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5310000000000003e-06 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14773 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030304+0000", 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011288212000000001, 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017016862000000001, 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14773 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030304+0000", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030304+0000", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030305+0000", 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:46.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030306+0000", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9640000000000002e-06 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030313+0000", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3960000000000003e-06 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030318+0000", 2024-09-17T21:47:46.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14937 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031992+0000", 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0096000800000000004, 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01538959, 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14937 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031992+0000", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031992+0000", 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031993+0000", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0500000000000003e-07 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031995+0000", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4619999999999999e-06 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032003+0000", 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2459999999999997e-06 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032013+0000", 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0753e-05 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15169 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032880+0000", 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0087119850000000002, 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014557675000000001, 2024-09-17T21:47:46.392 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15169 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032880+0000", 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032880+0000", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032881+0000", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4000000000000002e-07 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032883+0000", 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.877e-06 2024-09-17T21:47:46.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032889+0000", 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0050000000000001e-06 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032894+0000", 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8359999999999999e-06 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.395 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15248 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034808+0000", 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0067846549999999997, 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012689264, 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15248 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034808+0000", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034808+0000", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034808+0000", 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8999999999999996e-07 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034810+0000", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2529999999999999e-06 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034818+0000", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2989999999999999e-06 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034823+0000", 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3929999999999999e-06 2024-09-17T21:47:46.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15331 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037479+0000", 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0041138069999999997, 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010074048, 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15331 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037479+0000", 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037479+0000", 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038734+0000", 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001255756 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038739+0000", 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4299999999999999e-06 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038752+0000", 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2896e-05 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038760+0000", 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2239999999999995e-06 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15411 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.402 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042746+0000", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9988465, 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0048673329999999997, 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15411 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042746+0000", 2024-09-17T21:47:46.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042746+0000", 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042747+0000", 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7700000000000004e-07 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042749+0000", 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1380000000000001e-06 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042755+0000", 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3119999999999999e-06 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042760+0000", 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.933e-06 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14606 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.028706+0000", 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.012886498999999999, 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018968611999999999, 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14606 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028706+0000", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028706+0000", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028706+0000", 2024-09-17T21:47:46.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5300000000000004e-07 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028709+0000", 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4169999999999999e-06 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028719+0000", 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7219999999999994e-06 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.028736+0000", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7081000000000001e-05 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.047676+0000", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018940571999999999 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14774 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030324+0000", 2024-09-17T21:47:46.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011268449999999999, 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017424332000000001, 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14774 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030324+0000", 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030324+0000", 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030324+0000", 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2200000000000004e-07 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030326+0000", 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0109999999999999e-06 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030333+0000", 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6579999999999996e-06 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030345+0000", 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1768e-05 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.412 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14938 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032042+0000", 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0095507950000000008, 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015764838, 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14938 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032042+0000", 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032042+0000", 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032042+0000", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032045+0000", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7700000000000002e-06 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032054+0000", 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0880000000000001e-06 2024-09-17T21:47:46.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032061+0000", 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1930000000000004e-06 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15170 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032903+0000", 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0086893869999999998, 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014964511999999999, 2024-09-17T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15170 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032903+0000", 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032903+0000", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032903+0000", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7800000000000002e-07 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032905+0000", 2024-09-17T21:47:46.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.761e-06 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032911+0000", 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2299999999999996e-06 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032916+0000", 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7339999999999999e-06 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15249 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034834+0000", 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0067584619999999998, 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013089702999999999, 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15249 2024-09-17T21:47:46.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034834+0000", 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034834+0000", 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034834+0000", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9699999999999996e-07 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034837+0000", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2180000000000002e-06 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034844+0000", 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9099999999999999e-06 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034849+0000", 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1159999999999998e-06 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15332 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.038773+0000", 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0028196860000000001, 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0092110669999999999, 2024-09-17T21:47:46.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15332 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038773+0000", 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038773+0000", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038773+0000", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3600000000000003e-07 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038776+0000", 2024-09-17T21:47:46.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5009999999999999e-06 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038822+0000", 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.638e-05 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038829+0000", 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0129999999999996e-06 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15412 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042767+0000", 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9988256, 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0052760469999999999, 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15412 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042767+0000", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042767+0000", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042767+0000", 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-17T21:47:46.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042770+0000", 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1749999999999999e-06 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042776+0000", 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0549999999999996e-06 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042783+0000", 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.075e-06 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14775 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030351+0000", 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011240813000000001, 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017757763999999999, 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14775 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030351+0000", 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030351+0000", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030352+0000", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030354+0000", 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.906e-06 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030360+0000", 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030365+0000", 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.724e-06 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.433 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14939 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032070+0000", 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0095227030000000004, 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016098428000000001, 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14939 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032070+0000", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032070+0000", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032070+0000", 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032072+0000", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2589999999999999e-06 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032079+0000", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7970000000000003e-06 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032085+0000", 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8150000000000002e-06 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15171 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032922+0000", 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0086704659999999999, 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015305647, 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15171 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032922+0000", 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032922+0000", 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032922+0000", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032924+0000", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7749999999999999e-06 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032930+0000", 2024-09-17T21:47:46.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.998e-06 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032935+0000", 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5889999999999996e-06 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15250 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034855+0000", 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.006736962, 2024-09-17T21:47:46.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013438599000000001, 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15250 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034855+0000", 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034855+0000", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034856+0000", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034858+0000", 2024-09-17T21:47:46.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0480000000000001e-06 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034864+0000", 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034869+0000", 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15333 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.038850+0000", 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0027425589999999999, 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095202359999999996, 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15333 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038850+0000", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038850+0000", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038850+0000", 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1200000000000003e-07 2024-09-17T21:47:46.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038854+0000", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2040000000000002e-06 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038862+0000", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6500000000000002e-06 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038868+0000", 2024-09-17T21:47:46.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0419999999999999e-06 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15413 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042790+0000", 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9988022, 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005663347, 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15413 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042790+0000", 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042790+0000", 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042790+0000", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042803+0000", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2140999999999999e-05 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042809+0000", 2024-09-17T21:47:46.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.669e-06 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042815+0000", 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4689999999999996e-06 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14776 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030371+0000", 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011221768, 2024-09-17T21:47:46.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018172708999999999, 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14776 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030371+0000", 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030371+0000", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030371+0000", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4700000000000002e-07 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030373+0000", 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7799999999999999e-06 2024-09-17T21:47:46.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030379+0000", 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9490000000000001e-06 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030383+0000", 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.532e-06 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14948 3.1f 3:fb4f96ca:::rbd_data.11126af432c4.000000000000000e:25 [read 3772416~0] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032092+0000", 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0095005590000000004, 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016521661, 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14948 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032092+0000", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032092+0000", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032092+0000", 2024-09-17T21:47:46.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4499999999999997e-07 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032094+0000", 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9690000000000001e-06 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032101+0000", 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6229999999999999e-06 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032106+0000", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.048472+0000", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016366018 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.048538+0000", 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6561000000000002e-05 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15172 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032940+0000", 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0086523650000000004, 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015752566999999999, 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15172 2024-09-17T21:47:46.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032940+0000", 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032940+0000", 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032940+0000", 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032947+0000", 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3539999999999996e-06 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032953+0000", 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7549999999999997e-06 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032965+0000", 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1635e-05 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15251 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034877+0000", 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0067155069999999999, 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013887817, 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15251 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034877+0000", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034877+0000", 2024-09-17T21:47:46.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034877+0000", 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7999999999999996e-07 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034879+0000", 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1550000000000001e-06 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034887+0000", 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8080000000000005e-06 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034892+0000", 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1089999999999997e-06 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15334 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.464 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.038897+0000", 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0026950170000000001, 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0099295500000000005, 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15334 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038897+0000", 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038897+0000", 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038898+0000", 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3900000000000004e-07 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038901+0000", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6989999999999999e-06 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038910+0000", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3190000000000001e-06 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038924+0000", 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3837e-05 2024-09-17T21:47:46.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15414 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042823+0000", 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9987698, 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0060691110000000003, 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15414 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042823+0000", 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042823+0000", 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042823+0000", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7e-07 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042825+0000", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0250000000000001e-06 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042831+0000", 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:47:46.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042836+0000", 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7899999999999999e-06 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14865 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031216+0000", 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010376562000000001, 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017729847999999999, 2024-09-17T21:47:46.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14865 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031216+0000", 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031216+0000", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031216+0000", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9599999999999999e-07 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031219+0000", 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.373e-06 2024-09-17T21:47:46.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031226+0000", 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.362e-06 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031232+0000", 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.896e-06 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15177 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.032973+0000", 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0086198070000000002, 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016004475000000001, 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15177 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032973+0000", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032973+0000", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032973+0000", 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5200000000000002e-07 2024-09-17T21:47:46.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032975+0000", 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2249999999999999e-06 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032982+0000", 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4099999999999996e-06 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.032992+0000", 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0613e-05 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15259 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.034899+0000", 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.006693386, 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014106426, 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15259 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034899+0000", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.034899+0000", 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036703+0000", 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018045769999999999 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036708+0000", 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6770000000000003e-06 2024-09-17T21:47:46.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036722+0000", 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3674e-05 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036730+0000", 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3259999999999996e-06 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15343 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.038933+0000", 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0026597439999999999, 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010102013999999999, 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15343 2024-09-17T21:47:46.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038933+0000", 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038933+0000", 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038933+0000", 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8899999999999999e-07 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038936+0000", 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.441e-06 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038943+0000", 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0049999999999998e-06 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038948+0000", 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3059999999999997e-06 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15423 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042842+0000", 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9987497, 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0062199320000000001, 2024-09-17T21:47:46.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15423 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042842+0000", 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042842+0000", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042843+0000", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2399999999999999e-07 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042845+0000", 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9230000000000001e-06 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042851+0000", 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2940000000000002e-06 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042856+0000", 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.69e-06 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14866 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031241+0000", 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010351175000000001, 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017850903000000001, 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14866 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031241+0000", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031241+0000", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031242+0000", 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4199999999999996e-07 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031252+0000", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027e-05 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031260+0000", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7819999999999994e-06 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031269+0000", 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2799999999999992e-06 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15178 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033001+0000", 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0085915060000000005, 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016128401000000001, 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15178 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033001+0000", 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033001+0000", 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033001+0000", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.75e-07 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033003+0000", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2230000000000001e-06 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033011+0000", 2024-09-17T21:47:46.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1459999999999999e-06 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033016+0000", 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0710000000000002e-06 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15260 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036743+0000", 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0048491070000000001, 2024-09-17T21:47:46.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012416639, 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15260 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036743+0000", 2024-09-17T21:47:46.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036743+0000", 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036744+0000", 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.99e-07 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036746+0000", 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6249999999999999e-06 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036754+0000", 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7200000000000006e-06 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036760+0000", 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8590000000000001e-06 2024-09-17T21:47:46.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15344 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.038967+0000", 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0026253409999999998, 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010230927000000001, 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15344 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038967+0000", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038967+0000", 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038967+0000", 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038970+0000", 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.385e-06 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038977+0000", 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.289e-06 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.038982+0000", 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.276e-06 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15424 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042862+0000", 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9987307, 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064046210000000001, 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15424 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042862+0000", 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042862+0000", 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042862+0000", 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042864+0000", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8679999999999999e-06 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042870+0000", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8340000000000004e-06 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042875+0000", 2024-09-17T21:47:46.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7249999999999997e-06 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14867 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031278+0000", 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010314234, 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018050236000000001, 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14867 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031278+0000", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031278+0000", 2024-09-17T21:47:46.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031279+0000", 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-07 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031281+0000", 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4200000000000001e-06 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031288+0000", 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9190000000000002e-06 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031293+0000", 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2089999999999996e-06 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15179 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033023+0000", 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0085689560000000008, 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016362984000000001, 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15179 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033023+0000", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033023+0000", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033024+0000", 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.06e-07 2024-09-17T21:47:46.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033026+0000", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1210000000000001e-06 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033033+0000", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5470000000000002e-06 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033038+0000", 2024-09-17T21:47:46.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1030000000000001e-06 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15261 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036768+0000", 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0048242240000000002, 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012678495, 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15261 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036768+0000", 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036768+0000", 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036769+0000", 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6100000000000001e-07 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036771+0000", 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1390000000000002e-06 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036777+0000", 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5849999999999996e-06 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036782+0000", 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0699999999999997e-06 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15345 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039155+0000", 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0024370020000000002, 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01034769, 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15345 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039155+0000", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039155+0000", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039156+0000", 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8899999999999999e-07 2024-09-17T21:47:46.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039280+0000", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012370600000000001 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039291+0000", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1749000000000001e-05 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039298+0000", 2024-09-17T21:47:46.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9920000000000001e-06 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15425 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042880+0000", 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9987121, 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066847820000000002, 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15425 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042880+0000", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042880+0000", 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042881+0000", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042883+0000", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.883e-06 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042889+0000", 2024-09-17T21:47:46.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.178e-06 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042894+0000", 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1309999999999997e-06 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14868 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031301+0000", 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010290926000000001, 2024-09-17T21:47:46.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018319185000000002, 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14868 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031301+0000", 2024-09-17T21:47:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031301+0000", 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031302+0000", 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4499999999999997e-07 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031308+0000", 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2040000000000002e-06 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031315+0000", 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0230000000000004e-06 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031320+0000", 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1100000000000002e-06 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15180 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033051+0000", 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0085411489999999996, 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016630300000000001, 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15180 2024-09-17T21:47:46.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033051+0000", 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033051+0000", 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033052+0000", 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7199999999999999e-07 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033060+0000", 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4870000000000002e-06 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033068+0000", 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5140000000000004e-06 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033073+0000", 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2889999999999997e-06 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15262 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036799+0000", 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0047933380000000003, 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012948553999999999, 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.530 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15262 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036799+0000", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036799+0000", 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036800+0000", 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1900000000000002e-07 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036802+0000", 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4849999999999999e-06 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036809+0000", 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2359999999999998e-06 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036823+0000", 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3818000000000001e-05 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.533 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15346 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039311+0000", 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0022817229999999998, 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010652655, 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15346 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039311+0000", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039311+0000", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039311+0000", 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0399999999999995e-07 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039315+0000", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2310000000000001e-06 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039322+0000", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039328+0000", 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7640000000000002e-06 2024-09-17T21:47:46.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15426 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042900+0000", 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9986925, 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0071224299999999999, 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15426 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042900+0000", 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042900+0000", 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042900+0000", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042902+0000", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.933e-06 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042911+0000", 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2230000000000007e-06 2024-09-17T21:47:46.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042923+0000", 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2913e-05 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14793 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030513+0000", 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011079181, 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019572564000000001, 2024-09-17T21:47:46.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14793 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030513+0000", 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030513+0000", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030514+0000", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0600000000000001e-07 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030516+0000", 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9719999999999999e-06 2024-09-17T21:47:46.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030523+0000", 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.063e-06 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030527+0000", 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7870000000000001e-06 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15188 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033081+0000", 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0085111200000000005, 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017063983000000001, 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15188 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033081+0000", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033081+0000", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033082+0000", 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.51e-07 2024-09-17T21:47:46.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033084+0000", 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3209999999999999e-06 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033091+0000", 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5459999999999997e-06 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033095+0000", 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8409999999999999e-06 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15267 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036832+0000", 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0047599850000000004, 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013372828999999999, 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15267 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036832+0000", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036832+0000", 2024-09-17T21:47:46.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036833+0000", 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7299999999999995e-07 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036841+0000", 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8879999999999997e-06 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036849+0000", 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8290000000000008e-06 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036859+0000", 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0246e-05 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15351 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039335+0000", 2024-09-17T21:47:46.550 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0022568839999999998, 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010926353, 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15351 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039335+0000", 2024-09-17T21:47:46.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039335+0000", 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039336+0000", 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8400000000000005e-07 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039338+0000", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.266e-06 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039345+0000", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5039999999999999e-06 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039350+0000", 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2000000000000002e-06 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15435 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042931+0000", 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9986615, 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0073924660000000003, 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15435 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042931+0000", 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042931+0000", 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042931+0000", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042934+0000", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2800000000000002e-06 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042941+0000", 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8909999999999998e-06 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042946+0000", 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1619999999999999e-06 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14794 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030533+0000", 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011059391, 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019852479999999999, 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14794 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030533+0000", 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030533+0000", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030533+0000", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.39e-07 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030535+0000", 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.697e-06 2024-09-17T21:47:46.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030541+0000", 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9920000000000004e-06 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030554+0000", 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3138e-05 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15190 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033102+0000", 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084905269999999994, 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017343722999999998, 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15190 2024-09-17T21:47:46.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033102+0000", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033102+0000", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033102+0000", 2024-09-17T21:47:46.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8800000000000003e-07 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033104+0000", 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1399999999999998e-06 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033111+0000", 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4230000000000002e-06 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033117+0000", 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0469999999999999e-06 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15268 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036867+0000", 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0047252689999999998, 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013636463, 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.564 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15268 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036867+0000", 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036867+0000", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036868+0000", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0299999999999999e-07 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036870+0000", 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-06 2024-09-17T21:47:46.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036877+0000", 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0369999999999997e-06 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036882+0000", 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15352 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039357+0000", 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.002235187, 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011202999, 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15352 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039357+0000", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039357+0000", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039358+0000", 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9699999999999996e-07 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039367+0000", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8349999999999999e-06 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039375+0000", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9279999999999993e-06 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039381+0000", 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2769999999999996e-06 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15436 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042952+0000", 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.998641, 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0076733590000000003, 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15436 2024-09-17T21:47:46.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042952+0000", 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042952+0000", 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042952+0000", 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3099999999999998e-07 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042954+0000", 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.979e-06 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042960+0000", 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042965+0000", 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6380000000000003e-06 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14795 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030560+0000", 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011032174, 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020140956000000002, 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14795 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030560+0000", 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030560+0000", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030561+0000", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030562+0000", 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.874e-06 2024-09-17T21:47:46.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030569+0000", 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1850000000000001e-06 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030578+0000", 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0599999999999997e-06 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15191 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033123+0000", 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084688420000000007, 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017641562999999999, 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15191 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033123+0000", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033123+0000", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033124+0000", 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2199999999999999e-07 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033126+0000", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0470000000000001e-06 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033132+0000", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3029999999999996e-06 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033137+0000", 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8040000000000001e-06 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15269 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036889+0000", 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0047029020000000001, 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013933546999999999, 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15269 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036889+0000", 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036889+0000", 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036890+0000", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036892+0000", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9920000000000002e-06 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036898+0000", 2024-09-17T21:47:46.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2809999999999997e-06 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036903+0000", 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9289999999999997e-06 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15353 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039389+0000", 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0022032480000000001, 2024-09-17T21:47:46.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01149905, 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15353 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039389+0000", 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039389+0000", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039390+0000", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.82e-07 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039397+0000", 2024-09-17T21:47:46.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7780000000000007e-06 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039405+0000", 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7039999999999994e-06 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039410+0000", 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.096e-06 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15437 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042971+0000", 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.998621, 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079753180000000003, 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15437 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042971+0000", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042971+0000", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042972+0000", 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.32e-07 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042980+0000", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7680000000000008e-06 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042986+0000", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8709999999999999e-06 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042992+0000", 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.254e-06 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14796 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.030584+0000", 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.011008344999999999, 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020427976, 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14796 2024-09-17T21:47:46.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030584+0000", 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030584+0000", 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030585+0000", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030586+0000", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.973e-06 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030593+0000", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.618e-06 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.030598+0000", 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6580000000000001e-06 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15193 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033143+0000", 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084492790000000005, 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.017926975000000001, 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15193 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033143+0000", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033143+0000", 2024-09-17T21:47:46.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033144+0000", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4600000000000005e-07 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033146+0000", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.984e-06 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033153+0000", 2024-09-17T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033158+0000", 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15270 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036909+0000", 2024-09-17T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0046828750000000004, 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014221393000000001, 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15270 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036909+0000", 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036909+0000", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036910+0000", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036912+0000", 2024-09-17T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036918+0000", 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3899999999999998e-06 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036923+0000", 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9949999999999996e-06 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15354 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039451+0000", 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00214102, 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011738036, 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15354 2024-09-17T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039451+0000", 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039451+0000", 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039452+0000", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3099999999999997e-07 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039455+0000", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9110000000000002e-06 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039463+0000", 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0790000000000001e-06 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039489+0000", 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6228999999999999e-05 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15438 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.042999+0000", 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9985933, 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0082523149999999997, 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15438 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042999+0000", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.042999+0000", 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043000+0000", 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043002+0000", 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9659999999999999e-06 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043007+0000", 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8209999999999998e-06 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043012+0000", 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8250000000000004e-06 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14885 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031418+0000", 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010174135000000001, 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019893510999999999, 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14885 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031418+0000", 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031418+0000", 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031419+0000", 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3900000000000005e-07 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031421+0000", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1890000000000001e-06 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031428+0000", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031433+0000", 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0300000000000001e-06 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15197 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033165+0000", 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084276400000000001, 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018208906, 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15197 2024-09-17T21:47:46.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033165+0000", 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033165+0000", 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033165+0000", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1199999999999998e-07 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033167+0000", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0849999999999999e-06 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033174+0000", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4819999999999999e-06 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033178+0000", 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999996e-06 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15279 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036930+0000", 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0046627309999999998, 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014501211999999999, 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.614 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15279 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036930+0000", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036930+0000", 2024-09-17T21:47:46.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036930+0000", 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036932+0000", 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.81e-06 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036938+0000", 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1999999999999999e-06 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036943+0000", 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9200000000000003e-06 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15363 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.617 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039499+0000", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0020932799999999999, 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.011993913, 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15363 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039499+0000", 2024-09-17T21:47:46.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039499+0000", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039500+0000", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7199999999999999e-07 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039502+0000", 2024-09-17T21:47:46.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.841e-06 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039510+0000", 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7759999999999997e-06 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039522+0000", 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1601e-05 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15443 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043018+0000", 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9985747, 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0085338519999999998, 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15443 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043018+0000", 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043018+0000", 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043018+0000", 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3900000000000001e-07 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043019+0000", 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3239999999999999e-06 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043023+0000", 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4759999999999998e-06 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043025+0000", 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.401e-06 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14886 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031460+0000", 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010132459999999999, 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020153259999999999, 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14886 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031460+0000", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031460+0000", 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031460+0000", 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0999999999999998e-07 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031463+0000", 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4719999999999998e-06 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031471+0000", 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6510000000000001e-06 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031486+0000", 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5379e-05 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.627 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15198 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033185+0000", 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084074590000000008, 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018492062, 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15198 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033185+0000", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033185+0000", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033185+0000", 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033187+0000", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0619999999999999e-06 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033194+0000", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6490000000000002e-06 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033205+0000", 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1309e-05 2024-09-17T21:47:46.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15280 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036949+0000", 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0046430999999999998, 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.014789536000000001, 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15280 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036949+0000", 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036949+0000", 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036950+0000", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036952+0000", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0040000000000002e-06 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036958+0000", 2024-09-17T21:47:46.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5119999999999997e-06 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036963+0000", 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1329999999999998e-06 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15364 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039531+0000", 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.002061326, 2024-09-17T21:47:46.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012268935, 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15364 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039531+0000", 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039531+0000", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039532+0000", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7800000000000001e-07 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039534+0000", 2024-09-17T21:47:46.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.357e-06 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039541+0000", 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1949999999999997e-06 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039547+0000", 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2169999999999999e-06 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15444 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043028+0000", 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9985642, 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0088307230000000004, 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15444 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043028+0000", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043028+0000", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043028+0000", 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:46.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043030+0000", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1149999999999999e-06 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043035+0000", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5570000000000003e-06 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043040+0000", 2024-09-17T21:47:46.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4719999999999997e-06 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14887 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031494+0000", 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010098177999999999, 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020428992999999999, 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14887 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031494+0000", 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031494+0000", 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031495+0000", 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031497+0000", 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3099999999999999e-06 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031504+0000", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031524+0000", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0462000000000002e-05 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15199 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033212+0000", 2024-09-17T21:47:46.644 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0083803079999999995, 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.018768743000000001, 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15199 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033212+0000", 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033212+0000", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033212+0000", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033214+0000", 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0040000000000002e-06 2024-09-17T21:47:46.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033221+0000", 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0949999999999998e-06 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033226+0000", 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6809999999999997e-06 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15281 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036969+0000", 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0046231570000000001, 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015073071, 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15281 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036969+0000", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036969+0000", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036970+0000", 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036972+0000", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036978+0000", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2190000000000001e-06 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036983+0000", 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15365 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039554+0000", 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0020379399999999998, 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012546122, 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15365 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039554+0000", 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039554+0000", 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039555+0000", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039557+0000", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.024e-06 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039563+0000", 2024-09-17T21:47:46.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039568+0000", 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8629999999999999e-06 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15445 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043046+0000", 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9985466, 2024-09-17T21:47:46.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0091163000000000008, 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15445 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043046+0000", 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043046+0000", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043046+0000", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2500000000000001e-07 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043057+0000", 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1024e-05 2024-09-17T21:47:46.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043064+0000", 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9210000000000003e-06 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043069+0000", 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0769999999999998e-06 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:14888 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.031542+0000", 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.010050797, 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.020679932000000002, 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14888 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031542+0000", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031542+0000", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031542+0000", 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6700000000000003e-07 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031545+0000", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4449999999999999e-06 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031552+0000", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1409999999999999e-06 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.031557+0000", 2024-09-17T21:47:46.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0560000000000004e-06 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15200 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033232+0000", 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0083607449999999993, 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019048433, 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15200 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033232+0000", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033232+0000", 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033232+0000", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033238+0000", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7060000000000001e-06 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033244+0000", 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0759999999999999e-06 2024-09-17T21:47:46.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033248+0000", 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4680000000000003e-06 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15282 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.036989+0000", 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0046030910000000001, 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015345454999999999, 2024-09-17T21:47:46.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15282 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036989+0000", 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036989+0000", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036990+0000", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5499999999999998e-07 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036992+0000", 2024-09-17T21:47:46.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0090000000000002e-06 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.036998+0000", 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2199999999999997e-06 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037003+0000", 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7940000000000002e-06 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15366 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039574+0000", 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0020185229999999999, 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.012821618999999999, 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15366 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039574+0000", 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039574+0000", 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039574+0000", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039576+0000", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8250000000000001e-06 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039582+0000", 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9869999999999996e-06 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039587+0000", 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15446 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043077+0000", 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9985156, 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0093761209999999994, 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15446 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043077+0000", 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043077+0000", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043077+0000", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4299999999999998e-07 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043079+0000", 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:47:46.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043085+0000", 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3709999999999996e-06 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043097+0000", 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1746000000000001e-05 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15209 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033254+0000", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0083379530000000004, 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019259213000000001, 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15209 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033254+0000", 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033254+0000", 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033886+0000", 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000631892 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033890+0000", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2039999999999999e-06 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033902+0000", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1498000000000001e-05 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033909+0000", 2024-09-17T21:47:46.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15287 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037008+0000", 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.004584131, 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015562552, 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15287 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037008+0000", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037008+0000", 2024-09-17T21:47:46.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037009+0000", 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037010+0000", 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.784e-06 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037016+0000", 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037021+0000", 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6040000000000003e-06 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15371 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.681 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039621+0000", 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001971268, 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013008376, 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15371 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039621+0000", 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039621+0000", 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039622+0000", 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0999999999999999e-07 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039653+0000", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1585999999999999e-05 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039663+0000", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9380000000000004e-06 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039669+0000", 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:46.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15455 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043104+0000", 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9984884, 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0095838490000000002, 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.685 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15455 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043104+0000", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043104+0000", 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043104+0000", 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2599999999999998e-07 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043106+0000", 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2500000000000001e-06 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043113+0000", 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6660000000000002e-06 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043125+0000", 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1990999999999999e-05 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15210 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.688 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033944+0000", 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076481250000000004, 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01880799, 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15210 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033944+0000", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033944+0000", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033945+0000", 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4700000000000001e-07 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033948+0000", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8559999999999998e-06 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033956+0000", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6510000000000007e-06 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033963+0000", 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2310000000000001e-06 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15288 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037026+0000", 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.004566227, 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015787153000000002, 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15288 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.692 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037026+0000", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037026+0000", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037026+0000", 2024-09-17T21:47:46.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5600000000000001e-07 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037028+0000", 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7630000000000001e-06 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037034+0000", 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.772e-06 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037046+0000", 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1882000000000001e-05 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15372 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039688+0000", 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001904045, 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.01318222, 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.695 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15372 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039688+0000", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039688+0000", 2024-09-17T21:47:46.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039689+0000", 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2699999999999999e-07 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039692+0000", 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6340000000000001e-06 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039708+0000", 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6133999999999998e-05 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039714+0000", 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.964e-06 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15456 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.698 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043133+0000", 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9984598, 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0097990690000000005, 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15456 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043133+0000", 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043133+0000", 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043133+0000", 2024-09-17T21:47:46.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4200000000000001e-07 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043135+0000", 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2330000000000001e-06 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043142+0000", 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043147+0000", 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.78e-06 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15211 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.033972+0000", 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076202350000000004, 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.019018884, 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15211 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033972+0000", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033972+0000", 2024-09-17T21:47:46.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033973+0000", 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6000000000000004e-07 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033983+0000", 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0051000000000001e-05 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033990+0000", 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5939999999999996e-06 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.033996+0000", 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7699999999999998e-06 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15289 3.1f 3.5369f2df (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.037052+0000", 2024-09-17T21:47:46.705 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0045399569999999998, 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.015999227000000001, 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15289 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037052+0000", 2024-09-17T21:47:46.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037052+0000", 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037053+0000", 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037062+0000", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7489999999999993e-06 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037070+0000", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8960000000000003e-06 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.037076+0000", 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0979999999999999e-06 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15373 3.1b 3.1aa907bb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.039722+0000", 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.001870123, 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.013386360999999999, 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15373 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039722+0000", 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039722+0000", 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039723+0000", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3000000000000001e-07 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039725+0000", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.221e-06 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039731+0000", 2024-09-17T21:47:46.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1970000000000001e-06 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.039736+0000", 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8629999999999999e-06 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:15457 3.17 3.150a0bb7 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:46.712 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.043153+0000", 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9984393, 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010017511, 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15457 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043153+0000", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043153+0000", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043153+0000", 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0399999999999996e-07 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043155+0000", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9939999999999999e-06 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043161+0000", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.043166+0000", 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0379999999999999e-06 2024-09-17T21:47:46.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 189 2024-09-17T21:47:46.716 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:46.716 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:46.717 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-17T21:47:46.718 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:46.719 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:46.715+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:46.818 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:46.816+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:46.876 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:46.876 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:46.877 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:46.877 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:46.877 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:46.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:46.901 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:46.902 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:46.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:46.925 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:46.925 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:46.923+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:46.947 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:46.947 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:46.948 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:46.966 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:46.986 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:46.986 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:46.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:46.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:47.003 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:47:47.003 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:47.003 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:47:47.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:47:47.019 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:47.017+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:47.042 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.042 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:47.120 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:47.117+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:47.220 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:47.218+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:47.226 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.226 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:47.322 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:47.319+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 111, 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:47:45.994146+0000", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:47:26.936709+0000", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:47:06.858395+0000", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-17T21:47:47.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:47:47.386 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:47.387 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:47:47.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:47.389 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:47.390 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:47.391 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.392 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 5.880000114440918, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 5.880000114440918, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.17000000178813934, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.393 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:47.394 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:47:47.395 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:47.396 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:47.397 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.2699999809265137, 2024-09-17T21:47:47.398 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.2699999809265137, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:47:47.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:47:47.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "110", 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:47:47.401 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 39, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 110, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:47:47.402 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:47:47.403 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.404 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.5099999904632568, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.5099999904632568, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.82999998331069946, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.25, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.25, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:47:47.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 110, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.406 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.409 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.410 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:47:47.411 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:47:47.413 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.415 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:47:47.416 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.417 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.418 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:47:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.420 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:47.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:47:47.424 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 97, 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/2260220328", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/2260220328", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/2260220328", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/2260220328", 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-17T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 97, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:47:47.436 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:47.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:47:47.443 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.444 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 97 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:47.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T21:47:47.446 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:47.447 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:47.448 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:47.449 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.449 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.451 INFO:tasks.thrashosds.thrasher:j is {'epoch': 111, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:47:45.994146+0000', 'last_up_change': '2024-09-17T21:47:26.936709+0000', 'last_in_change': '2024-09-17T21:47:06.858395+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2699999809265137, 'score_stable': 2.2699999809265137, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 110, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5099999904632568, 'score_stable': 1.5099999904632568, 'optimal_score': 0.8299999833106995, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 110, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 0, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 81, 'down_at': 97, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 2260220328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 2260220328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 2260220328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 2260220328}]}, 'public_addr': '172.21.15.53:6817/2260220328', 'cluster_addr': '172.21.15.53:6819/2260220328', 'heartbeat_back_addr': '172.21.15.53:6823/2260220328', 'heartbeat_front_addr': '172.21.15.53:6821/2260220328', 'state': ['exists']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 97}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:47:47.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:47:47.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:47.452 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:47.447+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:47.550 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:47.548+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:47.625 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.625 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:47.651 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:47.649+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:47.751 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.751 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.752 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:47.749+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:47.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:47.852 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:47.850+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:47.855 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:47:47.856 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:47:47.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-17T21:47:46.118906+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1528797876,"num_objects":1441,"num_object_clones":270,"num_object_copies":2882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1441,"num_whiteouts":571,"num_read":188718,"num_read_kb":1183432,"num_write":14162,"num_write_kb":410672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2413,"num_bytes_recovered":336058168,"num_keys_recovered":522,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12167,"ondisk_log_size":12167,"up":67,"acting":67,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":868904,"kb_used_data":624876,"kb_used_omap":9,"kb_used_meta":243894,"kb_avail":565362136,"statfs":{"total":579820584960,"available":578930827264,"internally_reserved":0,"allocated":639873024,"data_stored":935545703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9568,"internal_metadata":249748128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9095256,"num_objects":102,"num_object_clones":8,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":87,"num_read":4178,"num_read_kb":5482,"num_write":129,"num_write_kb":4360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000587"},"pg_stats":[{"pgid":"3.1f","version":"106'230","reported_seq":4950,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.075561+0000","last_change":"2024-09-17T21:47:27.946117+0000","last_active":"2024-09-17T21:47:41.075561+0000","last_peered":"2024-09-17T21:47:41.075561+0000","last_clean":"2024-09-17T21:47:25.941505+0000","last_became_active":"2024-09-17T21:47:27.946117+0000","last_became_peered":"2024-09-17T21:47:27.946117+0000","last_unstale":"2024-09-17T21:47:41.075561+0000","last_undegraded":"2024-09-17T21:47:27.944784+0000","last_fullsized":"2024-09-17T21:47:27.944703+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:16:54.013938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":48439499,"num_objects":50,"num_object_clones":5,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":6248,"num_read_kb":59547,"num_write":215,"num_write_kb":9845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":10178260,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":50}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"106'397","reported_seq":9878,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.014078+0000","last_change":"2024-09-17T21:47:27.946014+0000","last_active":"2024-09-17T21:47:41.014078+0000","last_peered":"2024-09-17T21:47:41.014078+0000","last_clean":"2024-09-17T21:47:25.976109+0000","last_became_active":"2024-09-17T21:47:27.946014+0000","last_became_peered":"2024-09-17T21:47:27.946014+0000","last_unstale":"2024-09-17T21:47:41.014078+0000","last_undegraded":"2024-09-17T21:47:27.945209+0000","last_fullsized":"2024-09-17T21:47:27.945122+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:06.028877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":106674720,"num_objects":66,"num_object_clones":16,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":28,"num_read":10914,"num_read_kb":74352,"num_write":317,"num_write_kb":29710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":66}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"106'199","reported_seq":3575,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.405255+0000","last_change":"2024-09-17T21:47:10.889674+0000","last_active":"2024-09-17T21:47:40.405255+0000","last_peered":"2024-09-17T21:47:40.405255+0000","last_clean":"2024-09-17T21:47:40.405255+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:47:40.405255+0000","last_undegraded":"2024-09-17T21:47:40.405255+0000","last_fullsized":"2024-09-17T21:47:40.405255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:48:01.132679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":33237065,"num_objects":38,"num_object_clones":7,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":18,"num_read":3347,"num_read_kb":32401,"num_write":244,"num_write_kb":6809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"106'277","reported_seq":7548,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.019311+0000","last_change":"2024-09-17T21:47:11.890576+0000","last_active":"2024-09-17T21:47:41.019311+0000","last_peered":"2024-09-17T21:47:41.019311+0000","last_clean":"2024-09-17T21:47:41.019311+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:47:41.019311+0000","last_undegraded":"2024-09-17T21:47:41.019311+0000","last_fullsized":"2024-09-17T21:47:41.019311+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:36.529185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":71458917,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":13,"num_read":7090,"num_read_kb":57540,"num_write":355,"num_write_kb":34796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'321","reported_seq":5235,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.073121+0000","last_change":"2024-09-17T21:47:24.660656+0000","last_active":"2024-09-17T21:47:41.073121+0000","last_peered":"2024-09-17T21:47:41.073121+0000","last_clean":"2024-09-17T21:47:41.073121+0000","last_became_active":"2024-09-17T21:47:24.659839+0000","last_became_peered":"2024-09-17T21:47:24.659839+0000","last_unstale":"2024-09-17T21:47:41.073121+0000","last_undegraded":"2024-09-17T21:47:41.073121+0000","last_fullsized":"2024-09-17T21:47:41.073121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:57:08.992493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":63284752,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":6003,"num_read_kb":26888,"num_write":260,"num_write_kb":7463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":13117952,"num_keys_recovered":63,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"110'400","reported_seq":8038,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-17T21:47:45.072813+0000","last_change":"2024-09-17T21:47:10.889765+0000","last_active":"2024-09-17T21:47:45.072813+0000","last_peered":"2024-09-17T21:47:45.072813+0000","last_clean":"2024-09-17T21:47:45.072813+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:47:45.072813+0000","last_undegraded":"2024-09-17T21:47:45.072813+0000","last_fullsized":"2024-09-17T21:47:45.072813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:28.913623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":98054396,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":18,"num_read":9531,"num_read_kb":74730,"num_write":439,"num_write_kb":23147,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"106'299","reported_seq":3271,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.992580+0000","last_change":"2024-09-17T21:47:11.890115+0000","last_active":"2024-09-17T21:47:43.992580+0000","last_peered":"2024-09-17T21:47:43.992580+0000","last_clean":"2024-09-17T21:47:43.992580+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:47:43.992580+0000","last_undegraded":"2024-09-17T21:47:43.992580+0000","last_fullsized":"2024-09-17T21:47:43.992580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:36:14.390698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":33117314,"num_objects":41,"num_object_clones":6,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":18,"num_read":3228,"num_read_kb":17657,"num_write":320,"num_write_kb":8335,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"106'488","reported_seq":5624,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.074847+0000","last_change":"2024-09-17T21:46:56.829581+0000","last_active":"2024-09-17T21:47:41.074847+0000","last_peered":"2024-09-17T21:47:41.074847+0000","last_clean":"2024-09-17T21:47:41.074847+0000","last_became_active":"2024-09-17T21:46:56.828057+0000","last_became_peered":"2024-09-17T21:46:56.828057+0000","last_unstale":"2024-09-17T21:47:41.074847+0000","last_undegraded":"2024-09-17T21:47:41.074847+0000","last_fullsized":"2024-09-17T21:47:41.074847+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:48.956142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":74791673,"num_objects":49,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":5720,"num_read_kb":34829,"num_write":629,"num_write_kb":6657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":100,"num_bytes_recovered":4195404,"num_keys_recovered":48,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"106'210","reported_seq":2477,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.071746+0000","last_change":"2024-09-17T21:46:56.830587+0000","last_active":"2024-09-17T21:47:41.071746+0000","last_peered":"2024-09-17T21:47:41.071746+0000","last_clean":"2024-09-17T21:47:41.071746+0000","last_became_active":"2024-09-17T21:46:56.829387+0000","last_became_peered":"2024-09-17T21:46:56.829387+0000","last_unstale":"2024-09-17T21:47:41.071746+0000","last_undegraded":"2024-09-17T21:47:41.071746+0000","last_fullsized":"2024-09-17T21:47:41.071746+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:43.038370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":19136000,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":2299,"num_read_kb":25458,"num_write":223,"num_write_kb":7575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":89,"num_bytes_recovered":5990912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"106'636","reported_seq":4982,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:40.483262+0000","last_change":"2024-09-17T21:47:27.947066+0000","last_active":"2024-09-17T21:47:40.483262+0000","last_peered":"2024-09-17T21:47:40.483262+0000","last_clean":"2024-09-17T21:47:25.985753+0000","last_became_active":"2024-09-17T21:47:27.947066+0000","last_became_peered":"2024-09-17T21:47:27.947066+0000","last_unstale":"2024-09-17T21:47:40.483262+0000","last_undegraded":"2024-09-17T21:47:27.946448+0000","last_fullsized":"2024-09-17T21:47:27.946369+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:12:36.828936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":25669192,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":7525,"num_read_kb":44476,"num_write":808,"num_write_kb":14819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":53}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"106'519","reported_seq":7007,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.016773+0000","last_change":"2024-09-17T21:47:27.946159+0000","last_active":"2024-09-17T21:47:41.016773+0000","last_peered":"2024-09-17T21:47:41.016773+0000","last_clean":"2024-09-17T21:47:24.742016+0000","last_became_active":"2024-09-17T21:47:27.946159+0000","last_became_peered":"2024-09-17T21:47:27.946159+0000","last_unstale":"2024-09-17T21:47:41.016773+0000","last_undegraded":"2024-09-17T21:47:27.944238+0000","last_fullsized":"2024-09-17T21:47:27.944159+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:11:04.498858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":29674019,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":8998,"num_read_kb":28907,"num_write":481,"num_write_kb":12476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":5601792,"num_keys_recovered":35,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":48}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"110'219","reported_seq":5859,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-17T21:47:45.069805+0000","last_change":"2024-09-17T21:47:17.916575+0000","last_active":"2024-09-17T21:47:45.069805+0000","last_peered":"2024-09-17T21:47:45.069805+0000","last_clean":"2024-09-17T21:47:45.069805+0000","last_became_active":"2024-09-17T21:47:17.915249+0000","last_became_peered":"2024-09-17T21:47:17.915249+0000","last_unstale":"2024-09-17T21:47:45.069805+0000","last_undegraded":"2024-09-17T21:47:45.069805+0000","last_fullsized":"2024-09-17T21:47:45.069805+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:25:13.492024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":91075180,"num_objects":57,"num_object_clones":15,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":18,"num_read":5063,"num_read_kb":66693,"num_write":227,"num_write_kb":18240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":39957736,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":170,"reported_epoch":110,"state":"active+undersized","last_fresh":"2024-09-17T21:47:44.996288+0000","last_change":"2024-09-17T21:47:27.944405+0000","last_active":"2024-09-17T21:47:44.996288+0000","last_peered":"2024-09-17T21:47:44.996288+0000","last_clean":"2024-09-17T21:47:05.857035+0000","last_became_active":"2024-09-17T21:47:27.944405+0000","last_became_peered":"2024-09-17T21:47:27.944405+0000","last_unstale":"2024-09-17T21:47:44.996288+0000","last_undegraded":"2024-09-17T21:47:44.996288+0000","last_fullsized":"2024-09-17T21:47:27.943772+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:39.576192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"106'250","reported_seq":4783,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.933274+0000","last_change":"2024-09-17T21:46:56.829051+0000","last_active":"2024-09-17T21:47:40.933274+0000","last_peered":"2024-09-17T21:47:40.933274+0000","last_clean":"2024-09-17T21:47:40.933274+0000","last_became_active":"2024-09-17T21:46:56.828260+0000","last_became_peered":"2024-09-17T21:46:56.828260+0000","last_unstale":"2024-09-17T21:47:40.933274+0000","last_undegraded":"2024-09-17T21:47:40.933274+0000","last_fullsized":"2024-09-17T21:47:40.933274+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:15:53.386224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":37355712,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":4318,"num_read_kb":35175,"num_write":320,"num_write_kb":13772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":117,"num_bytes_recovered":12296387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":201,"reported_epoch":109,"state":"active+undersized","last_fresh":"2024-09-17T21:47:43.993278+0000","last_change":"2024-09-17T21:47:27.948536+0000","last_active":"2024-09-17T21:47:43.993278+0000","last_peered":"2024-09-17T21:47:43.993278+0000","last_clean":"2024-09-17T21:47:25.938548+0000","last_became_active":"2024-09-17T21:47:27.948536+0000","last_became_peered":"2024-09-17T21:47:27.948536+0000","last_unstale":"2024-09-17T21:47:43.993278+0000","last_undegraded":"2024-09-17T21:47:43.993278+0000","last_fullsized":"2024-09-17T21:47:27.947429+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:29:15.886471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"106'347","reported_seq":3395,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.074091+0000","last_change":"2024-09-17T21:46:03.270712+0000","last_active":"2024-09-17T21:47:41.074091+0000","last_peered":"2024-09-17T21:47:41.074091+0000","last_clean":"2024-09-17T21:47:41.074091+0000","last_became_active":"2024-09-17T21:46:01.265693+0000","last_became_peered":"2024-09-17T21:46:01.265693+0000","last_unstale":"2024-09-17T21:47:41.074091+0000","last_undegraded":"2024-09-17T21:47:41.074091+0000","last_fullsized":"2024-09-17T21:47:41.074091+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:09.300362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":43721299,"num_objects":52,"num_object_clones":12,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":22,"num_read":3490,"num_read_kb":30482,"num_write":424,"num_write_kb":9573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":106,"state":"active+undersized","last_fresh":"2024-09-17T21:47:39.981172+0000","last_change":"2024-09-17T21:47:27.945689+0000","last_active":"2024-09-17T21:47:39.981172+0000","last_peered":"2024-09-17T21:47:39.981172+0000","last_clean":"2024-09-17T21:46:51.807867+0000","last_became_active":"2024-09-17T21:47:27.945689+0000","last_became_peered":"2024-09-17T21:47:27.945689+0000","last_unstale":"2024-09-17T21:47:39.981172+0000","last_undegraded":"2024-09-17T21:47:39.981172+0000","last_fullsized":"2024-09-17T21:47:27.944013+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:43:22.806626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"106'436","reported_seq":9587,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.993132+0000","last_change":"2024-09-17T21:46:03.279109+0000","last_active":"2024-09-17T21:47:43.993132+0000","last_peered":"2024-09-17T21:47:43.993132+0000","last_clean":"2024-09-17T21:47:43.993132+0000","last_became_active":"2024-09-17T21:46:01.268063+0000","last_became_peered":"2024-09-17T21:46:01.268063+0000","last_unstale":"2024-09-17T21:47:43.993132+0000","last_undegraded":"2024-09-17T21:47:43.993132+0000","last_fullsized":"2024-09-17T21:47:43.993132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:02:09.320630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":73134080,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":19,"num_read":11819,"num_read_kb":44875,"num_write":316,"num_write_kb":9619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":224,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.993237+0000","last_change":"2024-09-17T21:46:53.661449+0000","last_active":"2024-09-17T21:47:43.993237+0000","last_peered":"2024-09-17T21:47:43.993237+0000","last_clean":"2024-09-17T21:47:43.993237+0000","last_became_active":"2024-09-17T21:46:53.661123+0000","last_became_peered":"2024-09-17T21:46:53.661123+0000","last_unstale":"2024-09-17T21:47:43.993237+0000","last_undegraded":"2024-09-17T21:47:43.993237+0000","last_fullsized":"2024-09-17T21:47:43.993237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:11:05.789195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"106'277","reported_seq":4391,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.071094+0000","last_change":"2024-09-17T21:46:03.282656+0000","last_active":"2024-09-17T21:47:41.071094+0000","last_peered":"2024-09-17T21:47:41.071094+0000","last_clean":"2024-09-17T21:47:41.071094+0000","last_became_active":"2024-09-17T21:46:01.271756+0000","last_became_peered":"2024-09-17T21:46:01.271756+0000","last_unstale":"2024-09-17T21:47:41.071094+0000","last_undegraded":"2024-09-17T21:47:41.071094+0000","last_fullsized":"2024-09-17T21:47:41.071094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:39:38.577765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":32515699,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":5123,"num_read_kb":24187,"num_write":224,"num_write_kb":6380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"110'608","reported_seq":5092,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-17T21:47:45.136490+0000","last_change":"2024-09-17T21:47:11.890882+0000","last_active":"2024-09-17T21:47:45.136490+0000","last_peered":"2024-09-17T21:47:45.136490+0000","last_clean":"2024-09-17T21:47:45.136490+0000","last_became_active":"2024-09-17T21:47:11.889462+0000","last_became_peered":"2024-09-17T21:47:11.889462+0000","last_unstale":"2024-09-17T21:47:45.136490+0000","last_undegraded":"2024-09-17T21:47:45.136490+0000","last_fullsized":"2024-09-17T21:47:45.136490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:07:55.345837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":29218755,"num_objects":42,"num_object_clones":6,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":6792,"num_read_kb":18251,"num_write":868,"num_write_kb":8332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":195,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:39.986004+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:47:39.986004+0000","last_peered":"2024-09-17T21:47:39.986004+0000","last_clean":"2024-09-17T21:47:39.986004+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:47:39.986004+0000","last_undegraded":"2024-09-17T21:47:39.986004+0000","last_fullsized":"2024-09-17T21:47:39.986004+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"106'208","reported_seq":2935,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.992899+0000","last_change":"2024-09-17T21:46:03.278873+0000","last_active":"2024-09-17T21:47:43.992899+0000","last_peered":"2024-09-17T21:47:43.992899+0000","last_clean":"2024-09-17T21:47:43.992899+0000","last_became_active":"2024-09-17T21:46:01.266414+0000","last_became_peered":"2024-09-17T21:46:01.266414+0000","last_unstale":"2024-09-17T21:47:43.992899+0000","last_undegraded":"2024-09-17T21:47:43.992899+0000","last_fullsized":"2024-09-17T21:47:43.992899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:27:36.025496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":24009851,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":18,"num_read":3399,"num_read_kb":15859,"num_write":193,"num_write_kb":4507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":193,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:39.980361+0000","last_change":"2024-09-17T21:45:54.241342+0000","last_active":"2024-09-17T21:47:39.980361+0000","last_peered":"2024-09-17T21:47:39.980361+0000","last_clean":"2024-09-17T21:47:39.980361+0000","last_became_active":"2024-09-17T21:45:52.236671+0000","last_became_peered":"2024-09-17T21:45:52.236671+0000","last_unstale":"2024-09-17T21:47:39.980361+0000","last_undegraded":"2024-09-17T21:47:39.980361+0000","last_fullsized":"2024-09-17T21:47:39.980361+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:46:29.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066043099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"106'1426","reported_seq":6999,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.788942+0000","last_change":"2024-09-17T21:47:11.891646+0000","last_active":"2024-09-17T21:47:40.788942+0000","last_peered":"2024-09-17T21:47:40.788942+0000","last_clean":"2024-09-17T21:47:40.788942+0000","last_became_active":"2024-09-17T21:47:11.890047+0000","last_became_peered":"2024-09-17T21:47:11.890047+0000","last_unstale":"2024-09-17T21:47:40.788942+0000","last_undegraded":"2024-09-17T21:47:40.788942+0000","last_fullsized":"2024-09-17T21:47:40.788942+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1426,"log_dups_size":0,"ondisk_log_size":1426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:51:35.658533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":42976084,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":18,"num_read":13166,"num_read_kb":52583,"num_write":2288,"num_write_kb":11295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":7135373,"num_keys_recovered":83,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":109,"state":"active+undersized","last_fresh":"2024-09-17T21:47:43.992952+0000","last_change":"2024-09-17T21:47:27.948863+0000","last_active":"2024-09-17T21:47:43.992952+0000","last_peered":"2024-09-17T21:47:43.992952+0000","last_clean":"2024-09-17T21:45:51.221520+0000","last_became_active":"2024-09-17T21:47:27.948863+0000","last_became_peered":"2024-09-17T21:47:27.948863+0000","last_unstale":"2024-09-17T21:47:43.992952+0000","last_undegraded":"2024-09-17T21:47:43.992952+0000","last_fullsized":"2024-09-17T21:47:27.947585+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:17:32.663327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"106'311","reported_seq":3838,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.992622+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:47:43.992622+0000","last_peered":"2024-09-17T21:47:43.992622+0000","last_clean":"2024-09-17T21:47:43.992622+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:47:43.992622+0000","last_undegraded":"2024-09-17T21:47:43.992622+0000","last_fullsized":"2024-09-17T21:47:43.992622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:58:02.599733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":35655230,"num_objects":39,"num_object_clones":5,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":18,"num_read":4296,"num_read_kb":26759,"num_write":354,"num_write_kb":17771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":194,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:39.986020+0000","last_change":"2024-09-17T21:45:54.240677+0000","last_active":"2024-09-17T21:47:39.986020+0000","last_peered":"2024-09-17T21:47:39.986020+0000","last_clean":"2024-09-17T21:47:39.986020+0000","last_became_active":"2024-09-17T21:45:52.236301+0000","last_became_peered":"2024-09-17T21:45:52.236301+0000","last_unstale":"2024-09-17T21:47:39.986020+0000","last_undegraded":"2024-09-17T21:47:39.986020+0000","last_fullsized":"2024-09-17T21:47:39.986020+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:26:11.239631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":324,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:39.980864+0000","last_change":"2024-09-17T21:47:27.946524+0000","last_active":"2024-09-17T21:47:39.980864+0000","last_peered":"2024-09-17T21:47:39.980864+0000","last_clean":"2024-09-17T21:46:51.807478+0000","last_became_active":"2024-09-17T21:47:27.946524+0000","last_became_peered":"2024-09-17T21:47:27.946524+0000","last_unstale":"2024-09-17T21:47:39.980864+0000","last_undegraded":"2024-09-17T21:47:27.944451+0000","last_fullsized":"2024-09-17T21:47:27.944375+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:17.630534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"106'330","reported_seq":3697,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.451857+0000","last_change":"2024-09-17T21:46:03.270987+0000","last_active":"2024-09-17T21:47:40.451857+0000","last_peered":"2024-09-17T21:47:40.451857+0000","last_clean":"2024-09-17T21:47:40.451857+0000","last_became_active":"2024-09-17T21:46:01.270323+0000","last_became_peered":"2024-09-17T21:46:01.270323+0000","last_unstale":"2024-09-17T21:47:40.451857+0000","last_undegraded":"2024-09-17T21:47:40.451857+0000","last_fullsized":"2024-09-17T21:47:40.451857+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:28.166559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":27779193,"num_objects":42,"num_object_clones":4,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":18,"num_read":4422,"num_read_kb":19864,"num_write":407,"num_write_kb":3622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"110'307","reported_seq":5608,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-09-17T21:47:44.996425+0000","last_change":"2024-09-17T21:47:11.890286+0000","last_active":"2024-09-17T21:47:44.996425+0000","last_peered":"2024-09-17T21:47:44.996425+0000","last_clean":"2024-09-17T21:47:44.996425+0000","last_became_active":"2024-09-17T21:47:11.889243+0000","last_became_peered":"2024-09-17T21:47:11.889243+0000","last_unstale":"2024-09-17T21:47:44.996425+0000","last_undegraded":"2024-09-17T21:47:44.996425+0000","last_fullsized":"2024-09-17T21:47:44.996425+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":307,"log_dups_size":0,"ondisk_log_size":307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:18:18.958456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":57963274,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":6216,"num_read_kb":52663,"num_write":308,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":34502270,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"110'305","reported_seq":2513,"reported_epoch":110,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:44.996303+0000","last_change":"2024-09-17T21:47:27.944271+0000","last_active":"2024-09-17T21:47:44.996303+0000","last_peered":"2024-09-17T21:47:44.996303+0000","last_clean":"2024-09-17T21:47:24.719749+0000","last_became_active":"2024-09-17T21:47:27.944271+0000","last_became_peered":"2024-09-17T21:47:27.944271+0000","last_unstale":"2024-09-17T21:47:44.996303+0000","last_undegraded":"2024-09-17T21:47:27.943873+0000","last_fullsized":"2024-09-17T21:47:27.943700+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":305,"log_dups_size":0,"ondisk_log_size":305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:44.216155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":7285376,"num_objects":30,"num_object_clones":4,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":2357,"num_read_kb":9790,"num_write":365,"num_write_kb":1564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":85,"num_bytes_recovered":721052,"num_keys_recovered":32,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":30}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"106'292","reported_seq":6161,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-17T21:47:43.993138+0000","last_change":"2024-09-17T21:46:56.829501+0000","last_active":"2024-09-17T21:47:43.993138+0000","last_peered":"2024-09-17T21:47:43.993138+0000","last_clean":"2024-09-17T21:47:43.993138+0000","last_became_active":"2024-09-17T21:46:56.828435+0000","last_became_peered":"2024-09-17T21:46:56.828435+0000","last_unstale":"2024-09-17T21:47:43.993138+0000","last_undegraded":"2024-09-17T21:47:43.993138+0000","last_fullsized":"2024-09-17T21:47:43.993138+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:37:26.171052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":32895564,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":18,"num_read":6945,"num_read_kb":40996,"num_write":276,"num_write_kb":9477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":13386408,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"106'355","reported_seq":3414,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:41.021796+0000","last_change":"2024-09-17T21:46:56.832225+0000","last_active":"2024-09-17T21:47:41.021796+0000","last_peered":"2024-09-17T21:47:41.021796+0000","last_clean":"2024-09-17T21:47:41.021796+0000","last_became_active":"2024-09-17T21:46:56.831587+0000","last_became_peered":"2024-09-17T21:46:56.831587+0000","last_unstale":"2024-09-17T21:47:41.021796+0000","last_undegraded":"2024-09-17T21:47:41.021796+0000","last_fullsized":"2024-09-17T21:47:41.021796+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:04.419303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":35623794,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":3296,"num_read_kb":22487,"num_write":384,"num_write_kb":9652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'606","reported_seq":8845,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:40.407919+0000","last_change":"2024-09-17T21:47:27.946463+0000","last_active":"2024-09-17T21:47:40.407919+0000","last_peered":"2024-09-17T21:47:40.407919+0000","last_clean":"2024-09-17T21:47:24.656822+0000","last_became_active":"2024-09-17T21:47:27.946463+0000","last_became_peered":"2024-09-17T21:47:27.946463+0000","last_unstale":"2024-09-17T21:47:40.407919+0000","last_undegraded":"2024-09-17T21:47:27.945275+0000","last_fullsized":"2024-09-17T21:47:27.945201+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":606,"log_dups_size":0,"ondisk_log_size":606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:40.344061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":115901535,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":14,"num_read":10207,"num_read_kb":67671,"num_write":676,"num_write_kb":26200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":47}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"106'221","reported_seq":2543,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.406667+0000","last_change":"2024-09-17T21:47:07.910393+0000","last_active":"2024-09-17T21:47:40.406667+0000","last_peered":"2024-09-17T21:47:40.406667+0000","last_clean":"2024-09-17T21:47:40.406667+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:47:40.406667+0000","last_undegraded":"2024-09-17T21:47:40.406667+0000","last_fullsized":"2024-09-17T21:47:40.406667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":221,"log_dups_size":0,"ondisk_log_size":221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:58:39.078870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":13899398,"num_objects":29,"num_object_clones":6,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":2365,"num_read_kb":12288,"num_write":279,"num_write_kb":5013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"106'514","reported_seq":7516,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.072402+0000","last_change":"2024-09-17T21:47:27.946920+0000","last_active":"2024-09-17T21:47:41.072402+0000","last_peered":"2024-09-17T21:47:41.072402+0000","last_clean":"2024-09-17T21:47:25.938891+0000","last_became_active":"2024-09-17T21:47:27.946920+0000","last_became_peered":"2024-09-17T21:47:27.946920+0000","last_unstale":"2024-09-17T21:47:41.072402+0000","last_undegraded":"2024-09-17T21:47:27.946398+0000","last_fullsized":"2024-09-17T21:47:27.946317+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:32:17.209760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":35025625,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":9884,"num_read_kb":53240,"num_write":687,"num_write_kb":27535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":47}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"106'385","reported_seq":6057,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-09-17T21:47:40.405003+0000","last_change":"2024-09-17T21:47:17.914088+0000","last_active":"2024-09-17T21:47:40.405003+0000","last_peered":"2024-09-17T21:47:40.405003+0000","last_clean":"2024-09-17T21:47:40.405003+0000","last_became_active":"2024-09-17T21:47:17.912983+0000","last_became_peered":"2024-09-17T21:47:17.912983+0000","last_unstale":"2024-09-17T21:47:40.405003+0000","last_undegraded":"2024-09-17T21:47:40.405003+0000","last_fullsized":"2024-09-17T21:47:40.405003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:43:54.383259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":34736959,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":12,"num_read":7344,"num_read_kb":20660,"num_write":389,"num_write_kb":4496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":4987381,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"110'281","reported_seq":4486,"reported_epoch":110,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:45.083221+0000","last_change":"2024-09-17T21:47:27.944406+0000","last_active":"2024-09-17T21:47:45.083221+0000","last_peered":"2024-09-17T21:47:45.083221+0000","last_clean":"2024-09-17T21:47:25.170760+0000","last_became_active":"2024-09-17T21:47:27.944406+0000","last_became_peered":"2024-09-17T21:47:27.944406+0000","last_unstale":"2024-09-17T21:47:45.083221+0000","last_undegraded":"2024-09-17T21:47:27.943759+0000","last_fullsized":"2024-09-17T21:47:27.943674+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:06.893812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":90261040,"num_objects":68,"num_object_clones":21,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":68,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":28,"num_read":3736,"num_read_kb":59281,"num_write":276,"num_write_kb":27769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":63162512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":68}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"106'192","reported_seq":997,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.022770+0000","last_change":"2024-09-17T21:47:27.946071+0000","last_active":"2024-09-17T21:47:41.022770+0000","last_peered":"2024-09-17T21:47:41.022770+0000","last_clean":"2024-09-17T21:47:25.941288+0000","last_became_active":"2024-09-17T21:47:27.946071+0000","last_became_peered":"2024-09-17T21:47:27.946071+0000","last_unstale":"2024-09-17T21:47:41.022770+0000","last_undegraded":"2024-09-17T21:47:27.944113+0000","last_fullsized":"2024-09-17T21:47:27.944030+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:39:50.327666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":25158732,"num_objects":27,"num_object_clones":3,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":740,"num_read_kb":8368,"num_write":213,"num_write_kb":4374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":3249740,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":27}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"106'201","reported_seq":2862,"reported_epoch":106,"state":"active+undersized+degraded","last_fresh":"2024-09-17T21:47:41.076984+0000","last_change":"2024-09-17T21:47:27.946645+0000","last_active":"2024-09-17T21:47:41.076984+0000","last_peered":"2024-09-17T21:47:41.076984+0000","last_clean":"2024-09-17T21:47:24.656637+0000","last_became_active":"2024-09-17T21:47:27.946645+0000","last_became_peered":"2024-09-17T21:47:27.946645+0000","last_unstale":"2024-09-17T21:47:41.076984+0000","last_undegraded":"2024-09-17T21:47:27.946169+0000","last_fullsized":"2024-09-17T21:47:27.946088+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:02:32.807005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":38477562,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":16,"num_read":2721,"num_read_kb":24375,"num_write":236,"num_write_kb":5875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":38}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1528207489,"num_objects":1437,"num_object_clones":270,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":571,"num_read":188602,"num_read_kb":1183332,"num_write":14001,"num_write_kb":408254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2407,"num_bytes_recovered":334287064,"num_keys_recovered":522,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":632315904,"data_stored":932336958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12047,"ondisk_log_size":12047,"up":54,"acting":54,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":1,"acting":1,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607579,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":151688,"kb_used_data":120500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94220152,"statfs":{"total":96636764160,"available":96481435648,"internally_reserved":0,"allocated":123392000,"data_stored":196561485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":12,"seq":51539607574,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148124,"kb_used_data":116936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94223716,"statfs":{"total":96636764160,"available":96485085184,"internally_reserved":0,"allocated":119742464,"data_stored":182624655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607578,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":143820,"kb_used_data":90424,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94228020,"statfs":{"total":96636764160,"available":96489492480,"internally_reserved":0,"allocated":92594176,"data_stored":139368968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607578,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":105984,"kb_used_data":70632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94265856,"statfs":{"total":96636764160,"available":96528236544,"internally_reserved":0,"allocated":72327168,"data_stored":89036676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607579,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":197308,"kb_used_data":139752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94174532,"statfs":{"total":96636764160,"available":96434720768,"internally_reserved":0,"allocated":143106048,"data_stored":206580215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640283,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121980,"kb_used_data":86632,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94249860,"statfs":{"total":96636764160,"available":96511856640,"internally_reserved":0,"allocated":88711168,"data_stored":121373704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71118848,"data_stored":88565512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87420928,"data_stored":120847198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91553792,"data_stored":139066284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":118235136,"data_stored":181733711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":141848576,"data_stored":206074367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":122138624,"data_stored":196049886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:47:47.871 INFO:tasks.thrashosds.thrasher:Setting 3.6 to [0, 5] 2024-09-17T21:47:47.871 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.6', '0', '5'] 2024-09-17T21:47:47.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.6 0 5 2024-09-17T21:47:47.903 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:47.903 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:47.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:47.953 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:47.950+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:48.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:47:48.062 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:48.060+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:48.095 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.095 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:48.155 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:48.152+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:48.224 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.224 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:48.276 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:48.274+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:48.356 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.357 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.357 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:48.354+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:48.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:48.457 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:48.455+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:48.513 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.513 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.522 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:48.558 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:48.556+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:48.637 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.637 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.646 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:48.659 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:48.657+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:48.753 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.753 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.760 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:48.757+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:48.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:48.861 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:48.859+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:48.928 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:48.928 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:48.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:48.980 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:48.978+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:48.991 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:49.012 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.6 pg_upmap mapping to [0,5] 2024-09-17T21:47:49.013 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:47:49.054 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:49.054 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:49.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:49.064 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:49.060+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:49.121 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:49.121 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:49.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:49.140 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:49.156 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:49.164 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:49.161+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:49.174 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:49.174 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:49.191 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:49.191 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:49.195 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:49.195 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:49.214 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:49.214 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:49.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:49.265 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:49.262+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:49.365 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:49.363+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:49.396 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.710606252000005, 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:49.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:49.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:49.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.678331276999998, 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:49.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:49.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:49.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:49.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.678092508999995, 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:49.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:49.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:49.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.510586379000003, 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:49.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:49.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.503684608, 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.421 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:49.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:49.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.503568516999998, 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:49.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12769 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3065888149999996, 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752021299999997, 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12769 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:49.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086842+0000", 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086847+0000", 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086852+0000", 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3920000000000002e-06 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086907+0000", 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5022999999999998e-05 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086919+0000", 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004339+0000", 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91742073199999996 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004359+0000", 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9558000000000001e-05 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3065649969999997, 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:47:49.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.441 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3065262019999997, 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:47:49.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:47:49.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.306511918, 2024-09-17T21:47:49.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:47:49.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:47:49.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3062669519999996, 2024-09-17T21:47:49.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:47:49.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:47:49.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.306184504, 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.456 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.460 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3061526939999997, 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.306128975, 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:49.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:47:49.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12809 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3059462059999998, 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752717500000003, 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12809 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:49.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087482+0000", 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8599999999999998e-07 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087485+0000", 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0649999999999999e-06 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087492+0000", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.187e-06 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087499+0000", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4609999999999996e-06 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087548+0000", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9926999999999998e-05 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087557+0000", 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005004+0000", 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744733599999995 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005008+0000", 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.941e-06 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12810 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3059166610000004, 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91753291699999995, 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12810 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:49.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087513+0000", 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087520+0000", 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087527+0000", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087583+0000", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6480999999999997e-05 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087590+0000", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9e-06 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005034+0000", 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744374799999995 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005044+0000", 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.696e-06 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12811 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3058933269999997, 2024-09-17T21:47:49.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752322500000005, 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12811 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:49.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087535+0000", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087537+0000", 2024-09-17T21:47:49.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087543+0000", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087549+0000", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6180000000000003e-06 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087716+0000", 2024-09-17T21:47:49.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016705100000000001 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087726+0000", 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5850000000000006e-06 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005053+0000", 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917327593 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005057+0000", 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0470000000000004e-06 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12812 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3058719100000005, 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91761393499999999, 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12812 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.92e-07 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087559+0000", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.26e-06 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087565+0000", 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-17T21:47:49.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087570+0000", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9459999999999997e-06 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087755+0000", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000185883 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087762+0000", 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7900000000000002e-06 2024-09-17T21:47:49.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005072+0000", 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917310179 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005169+0000", 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7089000000000005e-05 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12831 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087795+0000", 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3056327410000002, 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91743838099999997, 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12831 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087795+0000", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087795+0000", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087795+0000", 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087798+0000", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.39e-06 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087804+0000", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4049999999999997e-06 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087809+0000", 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9049999999999996e-06 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087867+0000", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8269000000000001e-05 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087874+0000", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4280000000000001e-06 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005224+0000", 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91735019100000004 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005233+0000", 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3549999999999994e-06 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12832 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087831+0000", 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.3055968900000003, 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744543700000003, 2024-09-17T21:47:49.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12832 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087831+0000", 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087831+0000", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087831+0000", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1099999999999996e-07 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087834+0000", 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7499999999999999e-06 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087842+0000", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1769999999999998e-06 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087847+0000", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4790000000000004e-06 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087897+0000", 2024-09-17T21:47:49.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9777999999999997e-05 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087904+0000", 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005266+0000", 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91736182600000005 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005276+0000", 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0241000000000001e-05 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:49.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:49.500 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:49.497+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:49.600 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:49.597+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:49.646 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:49.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:49.646 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:49.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:49.701 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:49.698+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:49.761 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:49.761 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:49.762 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:49.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:49.801 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:49.799+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:49.902 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:49.900+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012309119999998, 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:49.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:49.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012298818999994, 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:49.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012282744999993, 2024-09-17T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012271532, 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:49.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012254093999999, 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:49.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012243224000002, 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.012048347000004, 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:49.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.010343676999994, 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:49.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:49.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:49.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.010333039000002, 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:49.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.010322946000002, 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:49.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.010312827000007, 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:49.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:49.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.008182597000001, 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:49.973 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.008172865999995, 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:49.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.007389506999999, 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.978 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:49.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.007373936999997, 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:49.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.007281035000005, 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:49.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.006786415999997, 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.006642370999998, 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:49.997 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:49.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:49.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.005956933999997, 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.001 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:50.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.005594544000004, 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:50.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.010 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:50.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:50.012 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:50.009+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:50.112 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:50.110+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:50.119 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:50.119 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:50.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:50.213 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:50.211+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:50.276 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:50.276 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:50.276 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:50.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:50.314 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:50.311+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:50.415 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:50.412+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:50.433 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:50.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.557643025999994, 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:50.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:50.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:50.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:16776 3.18 3:1ae5f96e:::rbd_header.1112ffdab28d:head [watch reconnect cookie 140714604606752 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 10.350824229000001, 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021834859000000002, 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 16776 2024-09-17T21:47:50.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079591+0000", 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079592+0000", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079594+0000", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0999999999999998e-06 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079601+0000", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7139999999999996e-06 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.079607+0000", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9839999999999998e-06 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.096582+0000", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.016975463 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.096659+0000", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7359000000000003e-05 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:40.101426+0000", 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.004766803 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2564 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 702464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909038+0000", 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521377411, 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021662773999999999, 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2564 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909038+0000", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909038+0000", 2024-09-17T21:47:50.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909038+0000", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.23e-07 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909039+0000", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.037e-06 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909042+0000", 2024-09-17T21:47:50.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9859999999999999e-06 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909045+0000", 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930592+0000", 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021547133 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930630+0000", 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7938999999999997e-05 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930701+0000", 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0977999999999997e-05 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2565 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 700416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909049+0000", 2024-09-17T21:47:50.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521366322, 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021769713, 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2565 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909049+0000", 2024-09-17T21:47:50.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909049+0000", 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909049+0000", 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909051+0000", 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.186e-06 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909054+0000", 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.055e-06 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909056+0000", 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4090000000000001e-06 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930718+0000", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021661990999999998 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930754+0000", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6316999999999998e-05 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930819+0000", 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4530999999999996e-05 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2566 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 712704~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909060+0000", 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521355534, 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021875780000000001, 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2566 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909060+0000", 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909060+0000", 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909060+0000", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7799999999999997e-07 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909062+0000", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9300000000000002e-06 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909068+0000", 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3739999999999997e-06 2024-09-17T21:47:50.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909072+0000", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2860000000000001e-06 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930833+0000", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021761262999999999 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930872+0000", 2024-09-17T21:47:50.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8692000000000002e-05 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930936+0000", 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3956999999999998e-05 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2567 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 710656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909078+0000", 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521337242, 2024-09-17T21:47:50.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021975726000000001, 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2567 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909078+0000", 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909078+0000", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909079+0000", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.37e-07 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909085+0000", 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3080000000000004e-06 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909092+0000", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6730000000000003e-06 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909097+0000", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930949+0000", 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021852433000000001 2024-09-17T21:47:50.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.930987+0000", 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7805000000000001e-05 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931054+0000", 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6908000000000004e-05 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2568 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 722944~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909104+0000", 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521311796, 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02207081, 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2568 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909104+0000", 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909104+0000", 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909104+0000", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909106+0000", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1289999999999999e-06 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909112+0000", 2024-09-17T21:47:50.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7359999999999998e-06 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909117+0000", 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6090000000000003e-06 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931070+0000", 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.021952916999999999 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931110+0000", 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0144000000000001e-05 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931175+0000", 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4865999999999999e-05 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2569 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 733184~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909123+0000", 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5212929289999999, 2024-09-17T21:47:50.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022205341, 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2569 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909123+0000", 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909123+0000", 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909123+0000", 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5999999999999999e-07 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909125+0000", 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1009999999999999e-06 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909132+0000", 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7240000000000003e-06 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909136+0000", 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4299999999999999e-06 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931207+0000", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022070428 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931251+0000", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4622000000000001e-05 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931328+0000", 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6575999999999993e-05 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2570 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 720896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909143+0000", 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5212728580000001, 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022313487, 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2570 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909143+0000", 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909143+0000", 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909143+0000", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8799999999999998e-07 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909145+0000", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1849999999999998e-06 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909152+0000", 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3110000000000002e-06 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909157+0000", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9960000000000001e-06 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931353+0000", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022196806999999999 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931391+0000", 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7249000000000002e-05 2024-09-17T21:47:50.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931456+0000", 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5550999999999996e-05 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2571 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 743424~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909164+0000", 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.52125142, 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022423249999999999, 2024-09-17T21:47:50.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2571 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909164+0000", 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909164+0000", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909165+0000", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909167+0000", 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3750000000000001e-06 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909182+0000", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4939999999999999e-05 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909188+0000", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7919999999999998e-06 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931480+0000", 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022291999999999999 2024-09-17T21:47:50.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931518+0000", 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7936999999999999e-05 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931587+0000", 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9811e-05 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2572 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 731136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909196+0000", 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5212195570000002, 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022536796000000001, 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2572 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909196+0000", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909196+0000", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909196+0000", 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.01e-07 2024-09-17T21:47:50.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909199+0000", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.751e-06 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909208+0000", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3250000000000008e-06 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909213+0000", 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1220000000000003e-06 2024-09-17T21:47:50.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931612+0000", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022399239000000001 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931650+0000", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.858e-05 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931733+0000", 2024-09-17T21:47:50.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2378000000000003e-05 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5211967159999999, 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:47:50.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5211754150000001, 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:47:50.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5211546540000001, 2024-09-17T21:47:50.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:47:50.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5211335479999999, 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:47:50.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5211130590000002, 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:47:50.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5210889619999999, 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:50.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.521066469, 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:47:50.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:47:50.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5210388620000002, 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:47:50.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 3.5210140220000001, 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:50.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:50.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:47:50.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:47:50.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:50.525 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:50.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:50.527 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:50.524+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:50.628 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:50.626+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:50.646 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:50.647 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:50.647 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:50.647 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:50.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:50.728 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:50.726+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:50.763 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:50.763 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:50.772 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:50.829 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:50.827+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.924136177999998, 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:50.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.910150122000005, 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:50.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.908188914999997, 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:50.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:50.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.906155557999995, 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:50.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:50.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.90417764, 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:50.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:50.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:50.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.904154845999997, 2024-09-17T21:47:50.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:50.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.898245782999993, 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:50.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:50.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.896214724000004, 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:50.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:50.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:50.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.89618815, 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:50.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:50.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.895578357000005, 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.926 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:50.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:50.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.895560771000007, 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:50.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:50.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:50.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.895545814000002, 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:50.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:50.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.179864326000001, 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:50.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.179833704999993, 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:50.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.957 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.179809086999995, 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.958 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:50.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:50.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.179787140000002, 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:50.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:50.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 93.179765669999995, 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:50.970 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:50.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.138348867000005, 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:50.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:50.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 85.137304407000002, 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:50.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:50.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:50.987 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.936119076000001, 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:50.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:50.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:50.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:50.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:50.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:50.994 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:50.995 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:50.993+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:50.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:51.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:47:51.096 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:51.093+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:51.114 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:51.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:51.115 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:47:51.115 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:51.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:51.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:51.209 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:51.207+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:51.282 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:51.282 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:51.282 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:51.282 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:51.282 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:51.291 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:51.326 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:51.326 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:51.333 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:51.330+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:51.376 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:51.398 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:51.396+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:51.400 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:51.400 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:51.422 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:51.422 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:51.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:51.426 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:51.439 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:51.439 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:51.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:51.499 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:51.497+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:51.567 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:51.567 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:51.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:51.619 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:51.617+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:51.702 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:51.699+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:51.761 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:51.761 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:51.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:51.811 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:51.809+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:51.904 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:51.902+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:51.953 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:51.953 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:51.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:52.005 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.003+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:47:52.106 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:52.104+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:52.115 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.115 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.124 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:52.207 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:52.205+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:52.230 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.230 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:52.309 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.306+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.345 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.346 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:52.438 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.436+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.502 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:52.511 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.508+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.611 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.609+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.657 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.657 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:52.750 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:52.748+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:52.812 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:52.810+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:52.813 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.813 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.822 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:52.913 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:52.911+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:52.943 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:52.943 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:52.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:53.036 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:53.034+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:53.108 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:53.108 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:53.115 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:53.113+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:53.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:53.216 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:53.214+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:53.283 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:53.283 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:53.293 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:53.336 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:53.334+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:53.404 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:53.404 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:53.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:53.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:53.453 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:53.451+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:53.519 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:53.517+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:53.564 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:53.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:53.571 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:53.571 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:53.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:53.581 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:53.582 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:53.598 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:53.598 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:53.598 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:53.615 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:53.623 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:53.621+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:53.636 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:53.636 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:53.640 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:53.640 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:53.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722143931000005, 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:53.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722121438000002, 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:53.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:53.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722097177999999, 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:53.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:53.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722071295000006, 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:53.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:53.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722046293999995, 2024-09-17T21:47:53.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:53.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:53.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.722006495000002, 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.721990750000003, 2024-09-17T21:47:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.721979215000005, 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.721968915000005, 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719997672000005, 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719984523999997, 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719973917999994, 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719963579999998, 2024-09-17T21:47:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719953572999998, 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719943622000002, 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.719928314000001, 2024-09-17T21:47:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.717952427, 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.717941471000003, 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.717928982000004, 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 96.717918518999994, 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:53.786 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:53.787 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:53.785+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:53.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:53.888 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:53.886+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:53.915 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:53.916 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:53.916 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:53.916 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:53.925 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:53.989 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:53.986+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:54.030 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 0, 4] out_osds: [] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:47:54.030 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:47:54.030 INFO:tasks.thrashosds.thrasher:Removing osd 2, in_osds are: [1, 2, 3, 5, 0, 4] 2024-09-17T21:47:54.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-17T21:47:54.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:47:54.091 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:54.091 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:54.091 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:54.089+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:54.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:54.191 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:54.188+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:54.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.582650948999998, 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:54.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:54.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.550375974000005, 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:54.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.550137206000002, 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.382631076000003, 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:54.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:54.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.375729305, 2024-09-17T21:47:54.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:54.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:54.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 43.375613213999998, 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:54.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:54.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:54.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12769 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1786335119999993, 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752021299999997, 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12769 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086842+0000", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086847+0000", 2024-09-17T21:47:54.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086852+0000", 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3920000000000002e-06 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086907+0000", 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5022999999999998e-05 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086919+0000", 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004339+0000", 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91742073199999996 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004359+0000", 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9558000000000001e-05 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1786096940000004, 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:47:54.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:47:54.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1785708990000003, 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.314 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:47:54.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1785566149999998, 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:54.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1783116489999994, 2024-09-17T21:47:54.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:54.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1782292010000006, 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:47:54.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:54.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1781973909999994, 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:47:54.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:47:54.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:47:54.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1781736719999998, 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:47:54.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12809 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1779909029999995, 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752717500000003, 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.343 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12809 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087482+0000", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8599999999999998e-07 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087485+0000", 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0649999999999999e-06 2024-09-17T21:47:54.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087492+0000", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.187e-06 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087499+0000", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4609999999999996e-06 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087548+0000", 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9926999999999998e-05 2024-09-17T21:47:54.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087557+0000", 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005004+0000", 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744733599999995 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005008+0000", 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.941e-06 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12810 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1779613579999992, 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91753291699999995, 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12810 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:54.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087513+0000", 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087520+0000", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087527+0000", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087583+0000", 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6480999999999997e-05 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087590+0000", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9e-06 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005034+0000", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744374799999995 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005044+0000", 2024-09-17T21:47:54.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.696e-06 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12811 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1779380239999995, 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752322500000005, 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12811 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087535+0000", 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087537+0000", 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087543+0000", 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087549+0000", 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6180000000000003e-06 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087716+0000", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016705100000000001 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087726+0000", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5850000000000006e-06 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005053+0000", 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917327593 2024-09-17T21:47:54.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005057+0000", 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0470000000000004e-06 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12812 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1779166070000002, 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91761393499999999, 2024-09-17T21:47:54.358 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12812 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.92e-07 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087559+0000", 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.26e-06 2024-09-17T21:47:54.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087565+0000", 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087570+0000", 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9459999999999997e-06 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087755+0000", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000185883 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087762+0000", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7900000000000002e-06 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005072+0000", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917310179 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005169+0000", 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7089000000000005e-05 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7915342750000001, 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:47:54.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:47:54.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:54.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7816052950000003, 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:54.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:47:54.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:47:54.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.374 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:54.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:54.376 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:54.373+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:54.395 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:47:54.393+0000 7f0c28614640 -1 mon.a@0(leader).osd e116 definitely_dead 0 2024-09-17T21:47:54.476 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:54.474+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:54.509 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.509 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:54.509 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:54.509 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:54.515 INFO:teuthology.orchestra.run.smithi165.stdout:4279 read 0x408600 thru 0x40d285 (0x4c86 bytes) 2024-09-17T21:47:54.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:54.526 INFO:teuthology.orchestra.run.smithi165.stdout:4280 trunc from 0x6df2c00 to 0xd454400 2024-09-17T21:47:54.552 INFO:teuthology.orchestra.run.smithi165.stdout:4281 trunc from 0xd454400 to 0x88df800 2024-09-17T21:47:54.576 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:54.574+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:54.583 INFO:teuthology.orchestra.run.smithi165.stdout:4285 trunc from 0x88df800 to 0x2c09600 2024-09-17T21:47:54.610 INFO:teuthology.orchestra.run.smithi165.stdout:4286 read 0x16c8200 thru 0x16d68fa (0xe6fb bytes) 2024-09-17T21:47:54.615 INFO:teuthology.orchestra.run.smithi165.stdout:4287 clone 38 order 20 su 16384 sc 2 2024-09-17T21:47:54.678 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:54.675+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:54.680 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:54.680 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:54.680 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:54.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:54.779 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:54.776+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:54.797 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.797 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10701 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1138688~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890515742999995, 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055128254000000002, 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10701 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903851+0000", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903852+0000", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001e-06 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903855+0000", 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1659999999999998e-06 2024-09-17T21:47:54.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903858+0000", 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.543e-06 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958828+0000", 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054969927000000002 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958876+0000", 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7979999999999998e-05 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.958979+0000", 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103418 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10702 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1187840~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:54.802 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890505442000006, 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055298410999999999, 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10702 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903861+0000", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1899999999999999e-07 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903868+0000", 2024-09-17T21:47:54.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3579999999999999e-06 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903871+0000", 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5640000000000001e-06 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903874+0000", 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.486e-06 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959010+0000", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055136459999999998 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959055+0000", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4594e-05 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959159+0000", 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010473 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10703 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1236992~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890489368000004, 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055469496, 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10703 2024-09-17T21:47:54.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903877+0000", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903878+0000", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0979999999999999e-06 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903882+0000", 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1240000000000001e-06 2024-09-17T21:47:54.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903884+0000", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5179999999999999e-06 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959197+0000", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055312685 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959245+0000", 2024-09-17T21:47:54.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8049e-05 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959347+0000", 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101797 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10704 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1286144~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:54.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890478154999997, 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055638600000000003, 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10704 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903888+0000", 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.29e-07 2024-09-17T21:47:54.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903893+0000", 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5249999999999999e-06 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903900+0000", 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7739999999999999e-06 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903902+0000", 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6220000000000001e-06 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959377+0000", 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055475021999999999 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959424+0000", 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6137999999999998e-05 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959527+0000", 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010329 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10705 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1335296~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890460716999996, 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055810879000000001, 2024-09-17T21:47:54.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10705 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903906+0000", 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3999999999999998e-07 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903907+0000", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.11e-06 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903911+0000", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.233e-06 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903914+0000", 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5399999999999998e-06 2024-09-17T21:47:54.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959564+0000", 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055650070000000003 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959612+0000", 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.816e-05 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959717+0000", 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104526 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10706 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1384448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890449846999999, 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055980790000000002, 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10706 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.903917+0000", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904072+0000", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015515599999999999 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904075+0000", 2024-09-17T21:47:54.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2049999999999998e-06 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904085+0000", 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0049e-05 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904096+0000", 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.136e-05 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959748+0000", 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055651378000000001 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959792+0000", 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4011000000000003e-05 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959897+0000", 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010563100000000001 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10707 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1433600~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.890254970000001, 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055973795999999999, 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10707 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.904111+0000", 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905700+0000", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00158884 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905705+0000", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6059999999999996e-06 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905715+0000", 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0011e-05 2024-09-17T21:47:54.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905725+0000", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0278999999999999e-05 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959934+0000", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054208935 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.959982+0000", 2024-09-17T21:47:54.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8316999999999998e-05 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960085+0000", 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102808 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10711 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1630208~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.888550300000006, 2024-09-17T21:47:54.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055028085999999997, 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10711 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905816+0000", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-07 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905818+0000", 2024-09-17T21:47:54.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.203e-06 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905821+0000", 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4079999999999998e-06 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905824+0000", 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5349999999999999e-06 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960689+0000", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054865101999999999 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960739+0000", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0330000000000001e-05 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960844+0000", 2024-09-17T21:47:54.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010522900000000001 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10712 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1679360~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.888539661999999, 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055200608999999998, 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.835 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10712 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905827+0000", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1e-07 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905828+0000", 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2219999999999999e-06 2024-09-17T21:47:54.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905831+0000", 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1690000000000001e-06 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905834+0000", 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4669999999999998e-06 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960876+0000", 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055042512000000002 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.960922+0000", 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5932e-05 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961027+0000", 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104997 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10713 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1728512~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.888529568999999, 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055378854999999998, 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10713 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905837+0000", 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2999999999999999e-07 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905839+0000", 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.423e-06 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905842+0000", 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1310000000000002e-06 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905844+0000", 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4839999999999998e-06 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961066+0000", 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055221752999999998 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961113+0000", 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6813000000000001e-05 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961216+0000", 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103021 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.844 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10714 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 1777664~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.888519450000004, 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055564125999999998, 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10714 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.905847+0000", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907702+0000", 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001854788 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907706+0000", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4109999999999998e-06 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907716+0000", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5619999999999993e-06 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907722+0000", 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.691e-06 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961254+0000", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.053531502000000002 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961309+0000", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4635999999999997e-05 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.961411+0000", 2024-09-17T21:47:54.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102536 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10810 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 110592~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.886389219999998, 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055122418999999999, 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.849 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10810 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:54.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907977+0000", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0800000000000001e-07 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907979+0000", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.127e-06 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907982+0000", 2024-09-17T21:47:54.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1590000000000002e-06 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907984+0000", 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.962952+0000", 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054967685000000002 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963000+0000", 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8384000000000003e-05 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963100+0000", 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9533999999999999e-05 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10811 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 159744~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.886379489000007, 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055293491, 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10811 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:54.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.907987+0000", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908723+0000", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00073592600000000003 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908726+0000", 2024-09-17T21:47:54.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2430000000000001e-06 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908734+0000", 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6769999999999995e-06 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908738+0000", 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.707e-06 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963131+0000", 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054392428 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963176+0000", 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4805000000000002e-05 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963280+0000", 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010470499999999999 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10814 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 307200~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.885596129999996, 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055093731999999999, 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.858 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10814 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908770+0000", 2024-09-17T21:47:54.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908771+0000", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.34e-07 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908775+0000", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4190000000000004e-06 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908780+0000", 2024-09-17T21:47:54.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.865e-06 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908782+0000", 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6110000000000001e-06 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963712+0000", 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054929090999999999 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963758+0000", 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6499999999999999e-05 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963864+0000", 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106012 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10815 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 356352~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.885580559999994, 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055258464, 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10815 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908786+0000", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908842+0000", 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5723999999999997e-05 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908844+0000", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2000000000000001e-06 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908853+0000", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9209999999999998e-06 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908869+0000", 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5994e-05 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963895+0000", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055026654000000001 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.963941+0000", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5661999999999998e-05 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964044+0000", 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103309 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.866 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10816 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 405504~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.885487658000002, 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055354415999999997, 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10816 2024-09-17T21:47:54.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.908879+0000", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909352+0000", 2024-09-17T21:47:54.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000473444 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909355+0000", 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.106e-06 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909361+0000", 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9880000000000001e-06 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909365+0000", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0990000000000001e-06 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964081+0000", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054715591000000001 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964129+0000", 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7732000000000003e-05 2024-09-17T21:47:54.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964233+0000", 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104456 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10817 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 454656~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.884993038999994, 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055054084000000003, 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10817 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909373+0000", 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909480+0000", 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010690499999999999 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909484+0000", 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5599999999999998e-06 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909498+0000", 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-05 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909506+0000", 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.8040000000000007e-06 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964271+0000", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054764739 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964324+0000", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3207000000000001e-05 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964427+0000", 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103059 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10818 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 503808~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.884848993999995, 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.055095449999999997, 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10818 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909517+0000", 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909901+0000", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00038397399999999998 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909905+0000", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2509999999999999e-06 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909913+0000", 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5229999999999995e-06 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.909918+0000", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.374e-06 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964464+0000", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054546747 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964511+0000", 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6483000000000002e-05 2024-09-17T21:47:54.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.964613+0000", 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000102098 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10821 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 651264~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.884163556999994, 2024-09-17T21:47:54.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054962985999999998, 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10821 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910203+0000", 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910419+0000", 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000216307 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910423+0000", 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8999999999999999e-06 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910433+0000", 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8020000000000003e-06 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910440+0000", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5220000000000001e-06 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965019+0000", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054578566000000002 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965063+0000", 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4076000000000003e-05 2024-09-17T21:47:54.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965166+0000", 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010281300000000001 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4481.0:10823 3.2 3:4025fe03:::rbd_data.11125c054388.0000000000000005:7 [read 749568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.883801167000001, 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054970709999999999, 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:54.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4481", 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 10823 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910565+0000", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910802+0000", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023713900000000001 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910806+0000", 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910814+0000", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0059999999999992e-06 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.910823+0000", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3510000000000008e-06 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965384+0000", 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.054560299999999999 2024-09-17T21:47:54.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965430+0000", 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6396000000000001e-05 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:16.965536+0000", 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010584100000000001 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:54.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:54.891 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:54.892 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:54.889+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:54.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:54.992 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:54.990+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:54.999 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:54.999 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:54.999 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:55.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:55.026 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:47:55.029 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.2. 2024-09-17T21:47:55.093 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:55.091+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:55.136 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:55.136 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:55.137 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:55.137 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:55.145 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:55.194 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:55.192+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.253 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.377232061000001, 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.254 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:47:55.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:47:55.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3407857510000003, 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.260 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.34076445, 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:47:55.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.340743689, 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3407225829999998, 2024-09-17T21:47:55.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:47:55.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:47:55.277 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.278 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:55.279 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3407020939999992, 2024-09-17T21:47:55.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:47:55.279 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:47:55.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3406779970000002, 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:47:55.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:47:55.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3406555040000008, 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:47:55.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:47:55.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:47:55.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3406278969999992, 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:47:55.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:55.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.3406030569999992, 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:55.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:47:55.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1886492820000001, 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:47:55.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:47:55.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.306 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1886255200000004, 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1885831309999997, 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:47:55.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1328344780000004, 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:47:55.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:47:55.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1328113829999999, 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:47:55.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:47:55.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1327850030000004, 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:47:55.324 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:47:55.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:47:55.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1327683669999997, 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:47:55.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:47:55.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:47:55.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1327384030000003, 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:47:55.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1327254780000002, 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:47:55.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:47:55.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.1327152570000001, 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:55.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:55.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:47:55.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:55.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:55.347 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:55.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:55.349 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:55.346+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:55.449 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:55.446+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:55.463 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:1208 3.9 3.e27131e9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e117)", 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:55.035350+0000", 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.426695881, 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.42671719200000002, 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:55.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1208 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035350+0000", 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035350+0000", 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035351+0000", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9100000000000003e-07 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035354+0000", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8419999999999998e-06 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035365+0000", 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0488e-05 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035370+0000", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2320000000000001e-06 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035419+0000", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8633e-05 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:55.035427+0000", 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6340000000000007e-06 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.468 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:47:55.468 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:47:55.468 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:55.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:55.550 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:55.549+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:55.620 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:55.620 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:55.620 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:55.629 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:55.641 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:47:55.672 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:55.670+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:55.737 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:55.737 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:55.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.780782009000006, 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:55.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.766795952999999, 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:55.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:55.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:55.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:55.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.764834746000005, 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:55.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:55.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.762801389000003, 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.760823470999995, 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:55.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:55.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.760800677000006, 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:55.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.754891614000002, 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:55.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:55.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:55.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.752860554999998, 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:55.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:55.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.752833980999995, 2024-09-17T21:47:55.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:55.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:55.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:55.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.752224188, 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:55.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:55.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:55.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.752206602000001, 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:55.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.752191644999996, 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:55.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:55.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:55.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.036510156999995, 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:55.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:55.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:55.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.036479536000002, 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:55.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.810 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.036454918000004, 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:55.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.816 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.036432970999996, 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:55.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:55.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 98.036411501000003, 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:55.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:55.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.994994697999999, 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:55.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.993950237999996, 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:55.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:55.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.792764906999999, 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:55.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:55.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:55.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:55.846 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:55.847 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:55.847 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:55.848 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:55.848 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:55.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:55.849 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:55.849 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:55.849 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:55.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:55.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:55.850 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:55.851 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:55.851 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:55.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:55.851 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:55.845+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:55.948 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:55.946+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:55.996 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:55.996 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.004 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:56.049 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:56.047+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:56.113 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.113 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:56.150 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:56.148+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:56.228 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.228 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:56.251 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:56.248+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:56.346 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.346 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.351 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:56.349+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:56.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:56.452 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:56.450+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:56.501 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.501 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:56.554 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:56.552+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:56.617 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.617 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:56.654 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:56.652+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:56.755 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:56.753+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:56.774 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.774 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.784 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:56.857 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:56.854+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:56.890 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:56.891 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:56.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:56.957 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:56.954+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:57.007 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.007 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:57.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:47:57.057 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:57.055+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:57.148 INFO:teuthology.orchestra.run.smithi165.stdout:flattening image image_client.0-clone37 2024-09-17T21:47:57.152 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.152 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:47:57.186 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:57.156+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:57.259 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:57.257+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:57.360 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:57.357+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:57.363 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.363 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:47:57.460 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:57.458+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:57.561 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:57.559+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:57.592 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.592 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:47:57.662 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:57.660+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:57.761 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.761 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.763 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:57.761+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:57.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:47:57.852 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:47:57.897 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:57.895+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:57.965 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:57.963+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:57.968 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:47:57.968 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:47:57.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:47:58.001 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:47:58.001 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:47:58.018 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:47:58.018 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:47:58.035 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:47:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:47:58.035 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:47:58.044 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:47:58.060 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:47:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:47:58.084 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:47:58.085 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:47:58.085 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:58.082+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:58.089 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:47:58.089 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11001 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 872448~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:58.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.194514398, 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065295150999999996, 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11001 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.953182+0000", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954945+0000", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001763021 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954949+0000", 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5650000000000002e-06 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954956+0000", 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.627e-06 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.954964+0000", 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3640000000000001e-06 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018292+0000", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063328269000000006 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018339+0000", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6594999999999997e-05 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.018477+0000", 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013871000000000001 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11144 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1515520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.180528342, 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065262331000000007, 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11144 2024-09-17T21:47:58.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.967168+0000", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968946+0000", 2024-09-17T21:47:58.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001777434 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968950+0000", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.853e-06 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968963+0000", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2193e-05 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.968970+0000", 2024-09-17T21:47:58.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.187e-06 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032186+0000", 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063216623 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032245+0000", 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8629999999999999e-05 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.032431+0000", 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018541099999999999 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11151 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1859584~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.17856713499999, 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065308066999999997, 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11151 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.969129+0000", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970947+0000", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018176710000000001 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970951+0000", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3789999999999999e-06 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970964+0000", 2024-09-17T21:47:58.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2686e-05 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970971+0000", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2289999999999997e-06 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034182+0000", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063211039999999996 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034270+0000", 2024-09-17T21:47:58.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7607000000000003e-05 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.034437+0000", 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000167455 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11245 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 143360~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.17653377800001, 2024-09-17T21:47:58.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065363902000000002, 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11245 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.971163+0000", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972947+0000", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001784328 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972952+0000", 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972963+0000", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1352000000000001e-05 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036263+0000", 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063292577000000003 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036358+0000", 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4900000000000003e-05 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.036527+0000", 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000169085 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11253 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 487424~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.17455586, 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065395984000000004, 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11253 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:58.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973141+0000", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6699999999999999e-07 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973144+0000", 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6740000000000001e-06 2024-09-17T21:47:58.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973150+0000", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5590000000000002e-06 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973155+0000", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0089999999999998e-06 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038275+0000", 2024-09-17T21:47:58.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065120232 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038366+0000", 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0439000000000006e-05 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038537+0000", 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017070400000000001 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11254 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 536576~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.174533066, 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065579073000000002, 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11254 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973163+0000", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974945+0000", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781709 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974950+0000", 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974960+0000", 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974966+0000", 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038581+0000", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063614858999999996 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038640+0000", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9089999999999998e-05 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038743+0000", 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102253 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.177 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11277 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 1667072~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.16862400300001, 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065482835000000003, 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11277 2024-09-17T21:47:58.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.979073+0000", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980945+0000", 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018724620000000001 2024-09-17T21:47:58.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980949+0000", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2810000000000002e-06 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980957+0000", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1219999999999995e-06 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.980962+0000", 2024-09-17T21:47:58.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4089999999999996e-06 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044314+0000", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063351852 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044400+0000", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6343999999999999e-05 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.044555+0000", 2024-09-17T21:47:58.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000155365 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11285 3.9 3:939f92a8:::rbd_data.11125c054388.0000000000000003:7 [read 2060288~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.166592944, 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065354460000000003, 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11285 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981104+0000", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981106+0000", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981116+0000", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981123+0000", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9619999999999996e-06 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046212+0000", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065089312999999996 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046291+0000", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9429000000000001e-05 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046458+0000", 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000166625 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11374 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.16656637, 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067776232000000006, 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11374 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981130+0000", 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981700+0000", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057028500000000002 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981705+0000", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4360000000000004e-06 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981718+0000", 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3353000000000001e-05 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981726+0000", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5650000000000004e-06 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046196+0000", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064470628000000002 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046245+0000", 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8247000000000001e-05 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046456+0000", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021148200000000001 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048849+0000", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023933219999999998 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048858+0000", 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5269999999999998e-06 2024-09-17T21:47:58.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048884+0000", 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5786999999999999e-05 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048906+0000", 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.26e-05 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11375 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.165956577, 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067258681000000001, 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11375 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981740+0000", 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5599999999999995e-07 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981744+0000", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1389999999999999e-06 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981749+0000", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2290000000000002e-06 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981753+0000", 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7610000000000001e-06 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046950+0000", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065197275999999998 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.046999+0000", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8859000000000001e-05 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047202+0000", 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020319199999999999 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048914+0000", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001711777 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048948+0000", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4480000000000002e-05 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048978+0000", 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0158000000000002e-05 2024-09-17T21:47:58.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048999+0000", 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0254e-05 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11376 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.165938991, 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067327394999999998, 2024-09-17T21:47:58.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11376 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981758+0000", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999998e-07 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981760+0000", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.767e-06 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981763+0000", 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.428e-06 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981766+0000", 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.92e-06 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047585+0000", 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065819191999999999 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047630+0000", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299000000000001e-05 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.047824+0000", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019347200000000001 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049033+0000", 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093410000000001 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049040+0000", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4010000000000002e-06 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049064+0000", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4365000000000001e-05 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049085+0000", 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-05 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.165924034, 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:47:58.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14468 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:58.208 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.450242546, 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065614047999999994, 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14468 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697454+0000", 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697457+0000", 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7650000000000002e-06 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697463+0000", 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2389999999999999e-06 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697468+0000", 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8029999999999996e-06 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760005+0000", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.062536410000000001 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760093+0000", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8011999999999997e-05 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760406+0000", 2024-09-17T21:47:58.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031345400000000001 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762319+0000", 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001912923 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763003+0000", 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068376400000000001 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763040+0000", 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7129999999999999e-05 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763068+0000", 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8163000000000001e-05 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14469 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.450211925, 2024-09-17T21:47:58.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065737229999999994, 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14469 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697485+0000", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7599999999999997e-07 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697488+0000", 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2389999999999998e-06 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697496+0000", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000004e-06 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697501+0000", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5489999999999997e-06 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760806+0000", 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.063304689999999997 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.760891+0000", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4825000000000002e-05 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761203+0000", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031272 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763078+0000", 2024-09-17T21:47:58.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018742629999999999 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763124+0000", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6730000000000002e-05 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763183+0000", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8542999999999998e-05 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763222+0000", 2024-09-17T21:47:58.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8822000000000002e-05 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14470 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.45018730699999, 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.066864600999999996, 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14470 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697509+0000", 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697510+0000", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1699999999999999e-07 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697512+0000", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7030000000000002e-06 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697519+0000", 2024-09-17T21:47:58.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2060000000000004e-06 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697524+0000", 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0919999999999997e-06 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761632+0000", 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064108582999999997 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761695+0000", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3094000000000003e-05 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.761910+0000", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021459100000000001 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764243+0000", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023330299999999998 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764289+0000", 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6156999999999999e-05 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764335+0000", 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5562000000000002e-05 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.764374+0000", 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9165999999999997e-05 2024-09-17T21:47:58.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14471 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.45016536, 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068216219999999994, 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14471 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697531+0000", 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697532+0000", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697535+0000", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697541+0000", 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4200000000000004e-06 2024-09-17T21:47:58.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697546+0000", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9740000000000001e-06 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762353+0000", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.064806746999999998 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762406+0000", 2024-09-17T21:47:58.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2892000000000001e-05 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.762611+0000", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00020489799999999999 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765122+0000", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002511072 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765645+0000", 2024-09-17T21:47:58.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052353100000000004 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765697+0000", 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.215e-05 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765747+0000", 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9957999999999998e-05 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 100.45014389000001, 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:58.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:47:58.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:47:58.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 92.408727087000003, 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:47:58.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 92.407682627, 2024-09-17T21:47:58.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:47:58.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.206497295999998, 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:47:58.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:47:58.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.255 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:58.256 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:58.254+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:58.257 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:47:58.357 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:58.355+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:58.368 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15174 3.c 3.ad7bd94c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363614+0000", 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000456174, 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047706499999999999, 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15174 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363614+0000", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363614+0000", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363614+0000", 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2799999999999996e-07 2024-09-17T21:47:58.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363617+0000", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3029999999999998e-06 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9970000000000001e-06 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363629+0000", 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4609999999999999e-06 2024-09-17T21:47:58.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15175 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363637+0000", 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00043348699999999998, 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052036200000000002, 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15175 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363637+0000", 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363637+0000", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363637+0000", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999998e-07 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363639+0000", 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3910000000000001e-06 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363647+0000", 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2089999999999999e-06 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363652+0000", 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6060000000000002e-06 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15176 3.c 3.ad7bd94c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.375 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363821+0000", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00024861700000000002, 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039431900000000002, 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15176 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363821+0000", 2024-09-17T21:47:58.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363821+0000", 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363822+0000", 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7800000000000001e-07 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363826+0000", 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7179999999999998e-06 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363838+0000", 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1691999999999999e-05 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363846+0000", 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1100000000000003e-06 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15178 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363863+0000", 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00020702000000000001, 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040934499999999998, 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15178 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363863+0000", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363863+0000", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363864+0000", 2024-09-17T21:47:58.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999998e-07 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363867+0000", 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8789999999999999e-06 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363875+0000", 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6479999999999992e-06 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363882+0000", 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1629999999999999e-06 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15179 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363893+0000", 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00017677599999999999, 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00043596200000000002, 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15179 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363893+0000", 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363893+0000", 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363894+0000", 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363897+0000", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7700000000000002e-06 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363915+0000", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8936999999999998e-05 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363926+0000", 2024-09-17T21:47:58.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0601000000000001e-05 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15180 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363936+0000", 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.000134463, 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045047799999999998, 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15180 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363936+0000", 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363936+0000", 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363936+0000", 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3399999999999999e-07 2024-09-17T21:47:58.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363939+0000", 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0060000000000001e-06 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363953+0000", 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.416e-05 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363959+0000", 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.925e-06 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15181 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363968+0000", 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00010163299999999999, 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000474215, 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15181 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363968+0000", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363968+0000", 2024-09-17T21:47:58.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363969+0000", 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2399999999999998e-07 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363977+0000", 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7190000000000001e-06 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363984+0000", 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1289999999999998e-06 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363989+0000", 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1839999999999998e-06 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15182 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363998+0000", 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.2404000000000007e-05, 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050185300000000002, 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15182 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363998+0000", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363998+0000", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363998+0000", 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.46e-07 2024-09-17T21:47:58.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364001+0000", 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8490000000000002e-06 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364008+0000", 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8460000000000002e-06 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364015+0000", 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3180000000000001e-06 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15183 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364024+0000", 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6128999999999998e-05, 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00053204700000000001, 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.397 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15183 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364024+0000", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364024+0000", 2024-09-17T21:47:58.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364024+0000", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8599999999999998e-07 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364027+0000", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364033+0000", 2024-09-17T21:47:58.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364038+0000", 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2580000000000003e-06 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15184 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364046+0000", 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.4490000000000001e-05, 2024-09-17T21:47:58.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057866099999999995, 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15184 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364046+0000", 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364046+0000", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364046+0000", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0499999999999999e-07 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364048+0000", 2024-09-17T21:47:58.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1959999999999998e-06 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364055+0000", 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7710000000000001e-06 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364060+0000", 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15185 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364085+0000", 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999847, 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00060873899999999996, 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15185 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364085+0000", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364085+0000", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364086+0000", 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:47:58.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364089+0000", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9220000000000001e-06 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364096+0000", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6760000000000007e-06 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364102+0000", 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3510000000000001e-06 2024-09-17T21:47:58.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15186 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364112+0000", 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.999958, 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064197500000000003, 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.407 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15186 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364112+0000", 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364112+0000", 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364113+0000", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9999999999999998e-07 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364115+0000", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4420000000000001e-06 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364122+0000", 2024-09-17T21:47:58.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7880000000000001e-06 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364127+0000", 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1529999999999996e-06 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15187 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364134+0000", 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999361, 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00068131300000000001, 2024-09-17T21:47:58.410 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15187 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364134+0000", 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364134+0000", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364134+0000", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1399999999999997e-07 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364137+0000", 2024-09-17T21:47:58.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.728e-06 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364144+0000", 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5910000000000001e-06 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364149+0000", 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1200000000000001e-06 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15188 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364155+0000", 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9999146, 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072486800000000004, 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15188 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364155+0000", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364155+0000", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364156+0000", 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364158+0000", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2579999999999998e-06 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364164+0000", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1360000000000002e-06 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364169+0000", 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7400000000000004e-06 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15189 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364176+0000", 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9998941, 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075972699999999997, 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15189 2024-09-17T21:47:58.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364176+0000", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364176+0000", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364176+0000", 2024-09-17T21:47:58.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364178+0000", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.216e-06 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364185+0000", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3090000000000001e-06 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364190+0000", 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9660000000000004e-06 2024-09-17T21:47:58.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15190 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364196+0000", 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9998741, 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079934099999999996, 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15190 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364196+0000", 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364196+0000", 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364196+0000", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364198+0000", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.984e-06 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364205+0000", 2024-09-17T21:47:58.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4500000000000001e-06 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364210+0000", 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8760000000000004e-06 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15191 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364219+0000", 2024-09-17T21:47:58.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9998512, 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083113499999999999, 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15191 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364219+0000", 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364219+0000", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364219+0000", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6400000000000002e-07 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364222+0000", 2024-09-17T21:47:58.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4949999999999998e-06 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364229+0000", 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6340000000000003e-06 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364233+0000", 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.707e-06 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15192 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364240+0000", 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9998298, 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087247099999999999, 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15192 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364240+0000", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364240+0000", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364241+0000", 2024-09-17T21:47:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8100000000000003e-07 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364243+0000", 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0980000000000001e-06 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364250+0000", 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9809999999999997e-06 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364255+0000", 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9790000000000001e-06 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15193 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364262+0000", 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9998083, 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091354800000000003, 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15193 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364262+0000", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364262+0000", 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364262+0000", 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8500000000000002e-07 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364270+0000", 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1899999999999995e-06 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364277+0000", 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2099999999999996e-06 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364284+0000", 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15194 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364294+0000", 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9997764, 2024-09-17T21:47:58.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094125000000000003, 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15194 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364294+0000", 2024-09-17T21:47:58.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364294+0000", 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364294+0000", 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.46e-07 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364297+0000", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.553e-06 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364303+0000", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2829999999999998e-06 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364308+0000", 2024-09-17T21:47:58.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2379999999999997e-06 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15195 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364315+0000", 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9997554, 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097669899999999993, 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15195 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364315+0000", 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364315+0000", 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364315+0000", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364317+0000", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364324+0000", 2024-09-17T21:47:58.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364329+0000", 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1000000000000003e-06 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15196 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364336+0000", 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9997344, 2024-09-17T21:47:58.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010110570000000001, 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15196 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364336+0000", 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364336+0000", 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364336+0000", 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7300000000000002e-07 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364338+0000", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1390000000000002e-06 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364345+0000", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3400000000000003e-06 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364350+0000", 2024-09-17T21:47:58.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.95e-06 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15197 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364378+0000", 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.999692, 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010077720000000001, 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15197 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364378+0000", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364378+0000", 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364379+0000", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8299999999999997e-07 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364381+0000", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6620000000000001e-06 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364393+0000", 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1909999999999999e-05 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364399+0000", 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3190000000000002e-06 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15198 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364407+0000", 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9996629, 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010120050000000001, 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15198 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364407+0000", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364407+0000", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364408+0000", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8400000000000005e-07 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364410+0000", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4490000000000002e-06 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364417+0000", 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8120000000000002e-06 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364422+0000", 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0540000000000002e-06 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15199 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364430+0000", 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9996405, 2024-09-17T21:47:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010387899999999999, 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15199 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364430+0000", 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364430+0000", 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364430+0000", 2024-09-17T21:47:58.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7599999999999998e-07 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364432+0000", 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364439+0000", 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.139e-06 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364445+0000", 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0120000000000002e-06 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15200 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364451+0000", 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9996185, 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001073767, 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15200 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364451+0000", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364451+0000", 2024-09-17T21:47:58.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364452+0000", 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5199999999999998e-07 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364454+0000", 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0530000000000001e-06 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364460+0000", 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3770000000000001e-06 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364465+0000", 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9150000000000004e-06 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.457 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15201 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364472+0000", 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.999598, 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001110053, 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15201 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364472+0000", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364472+0000", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364472+0000", 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4799999999999999e-07 2024-09-17T21:47:58.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364475+0000", 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4449999999999999e-06 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364481+0000", 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.365e-06 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364487+0000", 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5450000000000003e-06 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15202 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364494+0000", 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9995766, 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001144691, 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.461 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15202 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364494+0000", 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364494+0000", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364494+0000", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5699999999999998e-07 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364496+0000", 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364501+0000", 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3529999999999997e-06 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364506+0000", 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.464 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15203 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364512+0000", 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9995575, 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001180784, 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15203 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364512+0000", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364512+0000", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364513+0000", 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7800000000000002e-07 2024-09-17T21:47:58.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364515+0000", 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5229999999999998e-06 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364522+0000", 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3090000000000001e-06 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364527+0000", 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1549999999999998e-06 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15204 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364534+0000", 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9995365, 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001197877, 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15204 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364534+0000", 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364534+0000", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364534+0000", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6100000000000002e-07 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364536+0000", 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364549+0000", 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2964e-05 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364555+0000", 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15205 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364563+0000", 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.999507, 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011986449999999999, 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15205 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364563+0000", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364563+0000", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364563+0000", 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6300000000000001e-07 2024-09-17T21:47:58.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364570+0000", 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9e-06 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364578+0000", 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3839999999999999e-06 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364583+0000", 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1370000000000001e-06 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15206 3.c 3.17f0ad4c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.364619+0000", 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4294967295.9994507, 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001173821, 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "queued for pg", 2024-09-17T21:47:58.475 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15206 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364619+0000", 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364619+0000", 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364620+0000", 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5099999999999998e-07 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364625+0000", 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4580000000000003e-06 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364636+0000", 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1399e-05 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.364644+0000", 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7889999999999995e-06 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:47:58.478 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 32 2024-09-17T21:47:58.479 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:58.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:47:58.481 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:58.477+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:58.581 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:58.578+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:58.627 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:58.627 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:47:58.627 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:47:58.627 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:47:58.627 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:58.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:47:58.682 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:47:58.679+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76996457, 2024-09-17T21:47:58.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:58.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:47:58.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769942077, 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:47:58.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76991781700001, 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.754 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769891934, 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:47:58.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769866933, 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:47:58.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769827134, 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:47:58.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769811389, 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.777 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769799854, 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:47:58.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.769789554, 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:47:58.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:47:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.767818311, 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:47:58.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76780516300001, 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:47:58.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.767794557, 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:58.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76778421900001, 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:58.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:47:58.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76777421200001, 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:47:58.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.767764261, 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76774895299999, 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:47:58.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76577306599999, 2024-09-17T21:47:58.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:47:58.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.76576211, 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:47:58.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.765749621, 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:47:58.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:47:58.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.765739158, 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:47:58.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:47:58.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:47:58.836 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:47:58.837 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:58.833+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:58.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:47:58.938 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:58.936+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14820 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.176077+0000", 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.76748978899999998, 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.76752747600000004, 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14820 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176077+0000", 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176077+0000", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176078+0000", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6799999999999999e-07 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176082+0000", 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4100000000000001e-06 2024-09-17T21:47:58.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176093+0000", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1216000000000001e-05 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176106+0000", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2683e-05 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176153+0000", 2024-09-17T21:47:58.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6635999999999999e-05 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176158+0000", 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.553e-06 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14821 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.176126+0000", 2024-09-17T21:47:58.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.76744015200000004, 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.76757896699999995, 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14821 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176126+0000", 2024-09-17T21:47:58.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176126+0000", 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176127+0000", 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2300000000000001e-07 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176130+0000", 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.185e-06 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176138+0000", 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2509999999999994e-06 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176144+0000", 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0100000000000001e-06 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176174+0000", 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.9441000000000002e-05 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176177+0000", 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.19e-06 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14881 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.176552+0000", 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.76701428400000005, 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.76719665500000001, 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14881 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176552+0000", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176552+0000", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176553+0000", 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176557+0000", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6540000000000001e-06 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176570+0000", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3485e-05 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176578+0000", 2024-09-17T21:47:58.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5689999999999999e-06 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176613+0000", 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4863000000000001e-05 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176616+0000", 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7689999999999998e-06 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14882 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.176593+0000", 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.76697328899999995, 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.76718926300000001, 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14882 2024-09-17T21:47:58.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176593+0000", 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176593+0000", 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176594+0000", 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9800000000000003e-07 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176597+0000", 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7130000000000001e-06 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176605+0000", 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9330000000000001e-06 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176611+0000", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176639+0000", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8390000000000001e-05 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.176642+0000", 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.061e-06 2024-09-17T21:47:58.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14950 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.178711+0000", 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.76485516200000003, 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.765104334, 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14950 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178711+0000", 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178711+0000", 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178712+0000", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2600000000000002e-07 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178716+0000", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7009999999999998e-06 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178725+0000", 2024-09-17T21:47:58.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1370000000000008e-06 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178732+0000", 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9689999999999997e-06 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178778+0000", 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.5791000000000003e-05 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178783+0000", 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8770000000000001e-06 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:14951 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.178751+0000", 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.764815731, 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.765097323, 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 14951 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178751+0000", 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178751+0000", 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178751+0000", 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9900000000000001e-07 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178754+0000", 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0910000000000001e-06 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178762+0000", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6340000000000001e-06 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178768+0000", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9460000000000003e-06 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178798+0000", 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0323000000000001e-05 2024-09-17T21:47:58.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.178803+0000", 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2799999999999997e-06 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15015 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.205675+0000", 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.737891238, 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.73820599099999995, 2024-09-17T21:47:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15015 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205675+0000", 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205675+0000", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205676+0000", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3799999999999996e-07 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205680+0000", 2024-09-17T21:47:58.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.67e-06 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205691+0000", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1362000000000001e-05 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205709+0000", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7604e-05 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205753+0000", 2024-09-17T21:47:58.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4576999999999997e-05 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205758+0000", 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7990000000000001e-06 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15016 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.205726+0000", 2024-09-17T21:47:58.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.73784090599999996, 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.73818941100000002, 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15016 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205726+0000", 2024-09-17T21:47:58.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205726+0000", 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205726+0000", 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7700000000000005e-07 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205729+0000", 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.79e-06 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205737+0000", 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0220000000000004e-06 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205743+0000", 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1519999999999997e-06 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205779+0000", 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5617999999999997e-05 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.205785+0000", 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0000000000000002e-06 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15080 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.223509+0000", 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.72005808999999998, 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.72043887200000001, 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15080 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223509+0000", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223509+0000", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223509+0000", 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0900000000000001e-07 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223513+0000", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6339999999999999e-06 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223524+0000", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1319e-05 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223532+0000", 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3429999999999998e-06 2024-09-17T21:47:58.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223576+0000", 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4169999999999999e-05 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223581+0000", 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8359999999999999e-06 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15081 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.223547+0000", 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.72001968599999999, 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.72043297100000003, 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15081 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223547+0000", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223547+0000", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223547+0000", 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3499999999999996e-07 2024-09-17T21:47:58.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223550+0000", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.852e-06 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223558+0000", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5240000000000003e-06 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223565+0000", 2024-09-17T21:47:58.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5839999999999997e-06 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223604+0000", 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8099999999999998e-05 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.223608+0000", 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.702e-06 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15153 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.363298+0000", 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.58026841900000004, 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.58071384699999995, 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15153 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363298+0000", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363298+0000", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363299+0000", 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1999999999999998e-07 2024-09-17T21:47:58.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363303+0000", 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0790000000000002e-06 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363314+0000", 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1358999999999999e-05 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363322+0000", 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2409999999999998e-06 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363370+0000", 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-05 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363375+0000", 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7650000000000001e-06 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:15154 3.18 3.6901a378 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.363342+0000", 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.58022508100000003, 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.58070305600000005, 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 15154 2024-09-17T21:47:58.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363342+0000", 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363342+0000", 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:58.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363342+0000", 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2099999999999997e-07 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363345+0000", 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7690000000000001e-06 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363358+0000", 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2865e-05 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363364+0000", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1759999999999998e-06 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363400+0000", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6239999999999999e-05 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.363406+0000", 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0650000000000004e-06 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:58.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:58.996 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:47:58.996 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 12 2024-09-17T21:47:58.996 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:58.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:47:59.039 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:59.037+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:59.140 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:59.137+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:59.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:47:59.240 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:59.238+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:59.338 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:59.338 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:59.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:59.338 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.652048668999996, 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:59.339 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:47:59.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:47:59.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.619773694000003, 2024-09-17T21:47:59.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.619534926, 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.350 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:47:59.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:59.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.452028796, 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.445127024999998, 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:47:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:47:59.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.445010934000003, 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:47:59.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:47:59.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:47:59.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12769 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.248031232000001, 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752021299999997, 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12769 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086842+0000", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086847+0000", 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:47:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086852+0000", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3920000000000002e-06 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086907+0000", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5022999999999998e-05 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086919+0000", 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T21:47:59.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004339+0000", 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91742073199999996 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004359+0000", 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9558000000000001e-05 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.375 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.248007414, 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:47:59.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247968619, 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.380 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:47:59.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247954334999999, 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.385 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:47:59.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:47:59.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247709369000001, 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.390 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:47:59.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:47:59.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247626921, 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:47:59.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:47:59.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247595111000001, 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.400 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247571391999999, 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:47:59.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12809 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247388623000001, 2024-09-17T21:47:59.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752717500000003, 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12809 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087482+0000", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8599999999999998e-07 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087485+0000", 2024-09-17T21:47:59.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0649999999999999e-06 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087492+0000", 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.187e-06 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087499+0000", 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4609999999999996e-06 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087548+0000", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9926999999999998e-05 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087557+0000", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005004+0000", 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744733599999995 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005008+0000", 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.941e-06 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12810 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247359078000001, 2024-09-17T21:47:59.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91753291699999995, 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12810 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087513+0000", 2024-09-17T21:47:59.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087520+0000", 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087527+0000", 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087583+0000", 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6480999999999997e-05 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087590+0000", 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9e-06 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005034+0000", 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744374799999995 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005044+0000", 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.696e-06 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12811 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247335744000001, 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752322500000005, 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12811 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087535+0000", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087537+0000", 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:47:59.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087543+0000", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087549+0000", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6180000000000003e-06 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087716+0000", 2024-09-17T21:47:59.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016705100000000001 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087726+0000", 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5850000000000006e-06 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005053+0000", 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917327593 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005057+0000", 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0470000000000004e-06 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12812 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.247314327, 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91761393499999999, 2024-09-17T21:47:59.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12812 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.92e-07 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087559+0000", 2024-09-17T21:47:59.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.26e-06 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087565+0000", 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087570+0000", 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9459999999999997e-06 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087755+0000", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000185883 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087762+0000", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7900000000000002e-06 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005072+0000", 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917310179 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005169+0000", 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7089000000000005e-05 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:47:59.429 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8609319949999996, 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:47:59.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:47:59.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.8510030149999999, 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:47:59.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:47:59.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.441 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:59.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:47:59.442 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:47:59.438+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:47:59.543 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:47:59.540+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:47:59.594 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:59.594 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:59.594 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:47:59.594 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:59.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:47:59.606 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-17T21:47:59.644 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:59.641+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:59.744 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:59.742+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:59.745 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:47:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:47:59.745 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:47:59.745 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:59.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:47:59.846 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:47:59.843+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:47:59.866 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:47:59.866 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:47:59.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:47:59.866 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.779793221, 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:47:59.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:47:59.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:47:59.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:47:59.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13835 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084165+0000", 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778672566, 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011545999999995, 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.872 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13835 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084165+0000", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084165+0000", 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084165+0000", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084171+0000", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9320000000000001e-06 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084177+0000", 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5940000000000001e-06 2024-09-17T21:47:59.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084181+0000", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3930000000000001e-06 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084637+0000", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000455523 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084640+0000", 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6430000000000001e-06 2024-09-17T21:47:59.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074273+0000", 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963278499999996 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074280+0000", 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2710000000000003e-06 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778650933, 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.877 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:47:59.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:47:59.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.881 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778634888, 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778618343, 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:47:59.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:47:59.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778603578, 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:47:59.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:47:59.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7785762730000001, 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778539385, 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778500687, 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7784625140000001, 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:47:59.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:47:59.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7784250849999999, 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:47:59.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:47:59.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778368062, 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:47:59.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7783373920000001, 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:47:59.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:47:59.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:47:59.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:47:59.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13851 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084625+0000", 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7782127910000001, 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011587000000001, 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13851 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084625+0000", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084625+0000", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084625+0000", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084627+0000", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9190000000000002e-06 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084633+0000", 2024-09-17T21:47:59.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.891e-06 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084638+0000", 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7600000000000002e-06 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085335+0000", 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00069704800000000003 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085342+0000", 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074733+0000", 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989391091 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074741+0000", 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2960000000000001e-06 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7781930190000002, 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:47:59.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:47:59.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:47:59.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:47:59.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7781741740000001, 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:47:59.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:47:59.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:47:59.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778147578, 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:47:59.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:47:59.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:47:59.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.778112739, 2024-09-17T21:47:59.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13856 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084752+0000", 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7780852810000001, 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011822500000002, 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13856 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084752+0000", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084752+0000", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:47:59.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9099999999999999e-07 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084755+0000", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1179999999999999e-06 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084761+0000", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4219999999999997e-06 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084767+0000", 2024-09-17T21:47:59.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5450000000000003e-06 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085549+0000", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078217900000000003 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085557+0000", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9610000000000005e-06 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074863+0000", 2024-09-17T21:47:59.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98930646700000002 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074870+0000", 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1420000000000004e-06 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13857 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084781+0000", 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.7780569640000001, 2024-09-17T21:47:59.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011857999999997, 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13857 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084781+0000", 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084781+0000", 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084781+0000", 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0900000000000002e-07 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:47:59.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084783+0000", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.294e-06 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084792+0000", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4789999999999996e-06 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084797+0000", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3759999999999999e-06 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085586+0000", 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00078932199999999996 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085590+0000", 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6629999999999999e-06 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074893+0000", 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98930239399999997 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:47:59.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074899+0000", 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6429999999999998e-06 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:47:59.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:47:59.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:47:59.962 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:47:59.958+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:00.048 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 0, 4] out_osds: [2] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:48:00.048 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:00.048 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [1, 3, 5, 0, 4] 2024-09-17T21:48:00.048 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-09-17T21:48:00.062 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:00.059+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:00.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:00.090 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:00.116 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:00.116 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:00.116 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:00.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:00.171 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:00.168+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:00.263 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:00.261+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:00.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:00.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:00.311 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:00.311 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:00.332 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:00.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:00.333 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:00.335 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:00.335 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:00.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:00.351 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:00.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:00.364 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:00.361+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:00.387 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:00.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:00.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:00.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:00.465 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:00.461+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:00.467 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:00.468 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:00.469 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:48:00.466+0000 7f0c28614640 -1 mon.a@0(leader).osd e121 definitely_dead 0 2024-09-17T21:48:00.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:00.566 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:00.562+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:00.587 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:00.587 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:00.596 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:00.667 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:00.664+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:00.724 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:00.724 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:00.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:00.768 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:00.764+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:00.869 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:00.865+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:00.895 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:00.896 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:00.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:00.970 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:00.967+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:01.034 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.034 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.043 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:01.086 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:01.084+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:01.169 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.169 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.172 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:01.169+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:01.178 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:01.180 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.0. 2024-09-17T21:48:01.181 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:48:01.273 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:01.270+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:01.326 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.326 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:01.378 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:01.375+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:01.442 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.443 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:01.492 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:01.489+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:01.575 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:01.572+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:01.598 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.598 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:01.691 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:01.688+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:01.755 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.755 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:01.777 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:01.774+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:01.877 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:01.875+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:01.910 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:01.911 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:01.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:01.978 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:01.975+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:02.026 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:02.026 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:02.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:02.079 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:02.076+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:02.179 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:02.176+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:02.186 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:02.187 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:02.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:02.280 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:02.278+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:02.343 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:02.343 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:02.352 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:02.381 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:02.379+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:02.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:02.460 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:02.460 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:02.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:02.482 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:02.479+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:02.520 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:02.521 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:02.538 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:02.538 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:02.554 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:02.555 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:02.572 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:02.572 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:02.579 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:02.579 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.703848128999994, 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:02.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:02.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:02.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.586 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667401819, 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:02.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667380518, 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:02.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:02.595 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667359757, 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:02.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:02.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667338651, 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:02.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667318162000001, 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:02.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:02.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667294065, 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:02.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:02.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667271572000001, 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:02.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:02.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667243965000001, 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.619 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:02.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.623 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.667219125000001, 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.624 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:02.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:02.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.51526535, 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.629 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:02.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.515241588, 2024-09-17T21:48:02.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:02.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:02.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.515199199, 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.638 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:02.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:02.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459450545999999, 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.643 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:02.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459427451, 2024-09-17T21:48:02.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459401071, 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:02.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459384435, 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:02.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:02.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459354470999999, 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.662 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:02.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:02.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459341545999999, 2024-09-17T21:48:02.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:02.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 11.459331325000001, 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:02.671 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:02.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:02.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:02.675 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:02.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:02.677 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:02.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:02.678 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:02.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:02.679 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:02.674+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:02.777 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:02.775+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:02.799 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:02.799 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:02.799 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:02.808 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:02.878 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:02.875+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:02.957 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:02.957 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:02.958 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:02.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:03.009 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:03.007+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:03.074 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:03.074 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11377 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.089224148, 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068564106999999999, 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11377 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.981772+0000", 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982946+0000", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174001 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982952+0000", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0289999999999996e-06 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982962+0000", 2024-09-17T21:48:03.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0536999999999999e-05 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.982968+0000", 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7420000000000003e-06 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048183+0000", 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065215023999999996 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048229+0000", 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5886000000000001e-05 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.048469+0000", 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024048299999999999 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.049092+0000", 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062329400000000004 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050268+0000", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011750630000000001 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050301+0000", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3609999999999998e-05 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.050337+0000", 2024-09-17T21:48:03.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5438000000000002e-05 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:14472 3.1 3:845283c9:::rbd_object_map.11121f8e87bb:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 105.37344400400001, 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068316476000000001, 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:03.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14472 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:48:03.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697553+0000", 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697556+0000", 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5959999999999998e-06 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697562+0000", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1720000000000004e-06 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.697567+0000", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8600000000000001e-06 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763273+0000", 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.065706159 2024-09-17T21:48:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763347+0000", 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4249e-05 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.763718+0000", 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037062999999999999 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:03.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765762+0000", 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020447109999999998 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765791+0000", 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8308000000000001e-05 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765832+0000", 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1230999999999999e-05 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.765869+0000", 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7156000000000001e-05 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.332027201000002, 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.087 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:03.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:03.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.330982741, 2024-09-17T21:48:03.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:03.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:03.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.129797410000002, 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:03.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:03.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:03.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.9138797419999998, 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:03.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:03.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.711443, 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:03.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:03.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:03.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7113858200000003, 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:03.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7108152390000004, 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:03.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:03.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7107873050000002, 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:03.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7106529530000003, 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:03.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:03.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7106314180000002, 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:03.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:03.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7104704530000001, 2024-09-17T21:48:03.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:03.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:03.155 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7104146880000002, 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.156 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:03.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7090680809999999, 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.709016525, 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7083282640000004, 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.708311213, 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.707372973, 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7073426329999997, 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.194 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:03.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:03.196 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:03.192+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:03.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:48:03.295 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:03.293+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:03.392 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:03.392 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:03.392 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:03.392 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:03.396 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:03.394+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:03.404 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:03.497 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:03.494+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:03.584 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:03.585 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:03.585 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:03.585 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:03.585 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:03.594 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:03.597 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:03.594+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:03.659 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:48:03.718 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:03.715+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75305706899999, 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.753034576, 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:03.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:03.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.753010316, 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:03.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:03.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.740 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75298443299999, 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.752959432, 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:03.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.752919633, 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:03.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:03.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75290388800001, 2024-09-17T21:48:03.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:03.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:03.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75289235299999, 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.763 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75288205299999, 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:03.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75091080999999, 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.768 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:03.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.750897662, 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:03.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:03.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.777 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.750887056, 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:03.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.750876718, 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.750866711, 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.75085676, 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:03.791 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:03.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.750841452, 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:03.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:03.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.748865565, 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:03.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:03.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:03.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.74885460900001, 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:03.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.74884212000001, 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:03.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 106.748831657, 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:03.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:03.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:03.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:03.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:03.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:03.818 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:03.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:03.820 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:03.816+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:03.920 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:03.917+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:03.936 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:04.021 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:04.018+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:04.084 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:04.084 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:04.084 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:04.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:04.133 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:04.130+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:04.223 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:04.220+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:04.241 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:04.241 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 102.555465895, 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:04.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:04.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:04.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 102.52319092, 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.252 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 102.522952152, 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:04.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:04.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.355446022000002, 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:04.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.348544251, 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:04.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.348428159999997, 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:04.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12769 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151448458000001, 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752021299999997, 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12769 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086839+0000", 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086842+0000", 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.48e-06 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086847+0000", 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0599999999999998e-06 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086852+0000", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3920000000000002e-06 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086907+0000", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5022999999999998e-05 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086919+0000", 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1564e-05 2024-09-17T21:48:04.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004339+0000", 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91742073199999996 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004359+0000", 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9558000000000001e-05 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151424639999998, 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.283 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:04.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151385845, 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:04.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151371561000001, 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:04.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:04.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151126595000001, 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:04.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151044147, 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:04.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.151012337000001, 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:04.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:04.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:04.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.150988618, 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:04.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:04.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12809 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.150805849000001, 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752717500000003, 2024-09-17T21:48:04.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12809 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:48:04.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087481+0000", 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087482+0000", 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8599999999999998e-07 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087485+0000", 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0649999999999999e-06 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087492+0000", 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.187e-06 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087499+0000", 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4609999999999996e-06 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087548+0000", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9926999999999998e-05 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087557+0000", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.6719999999999993e-06 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005004+0000", 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744733599999995 2024-09-17T21:48:04.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005008+0000", 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.941e-06 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12810 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.150776304000001, 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91753291699999995, 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12810 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087511+0000", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-07 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087513+0000", 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:04.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087520+0000", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.917e-06 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087527+0000", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087583+0000", 2024-09-17T21:48:04.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6480999999999997e-05 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087590+0000", 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9e-06 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005034+0000", 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91744374799999995 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005044+0000", 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.696e-06 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12811 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.150752969999999, 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91752322500000005, 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12811 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087534+0000", 2024-09-17T21:48:04.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087535+0000", 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3599999999999999e-07 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087537+0000", 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.322e-06 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087543+0000", 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5729999999999996e-06 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087549+0000", 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6180000000000003e-06 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087716+0000", 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00016705100000000001 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087726+0000", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5850000000000006e-06 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005053+0000", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917327593 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005057+0000", 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0470000000000004e-06 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12812 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.150731553, 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91761393499999999, 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12812 2024-09-17T21:48:04.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087556+0000", 2024-09-17T21:48:04.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.92e-07 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087559+0000", 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.26e-06 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087565+0000", 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0959999999999997e-06 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087570+0000", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9459999999999997e-06 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087755+0000", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000185883 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087762+0000", 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7900000000000002e-06 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005072+0000", 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917310179 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.005169+0000", 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7089000000000005e-05 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.764349221, 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.337 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:04.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.754420241, 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:04.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.347 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:04.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:04.349 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:04.346+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:04.449 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:04.446+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:04.497 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.497 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3.7f293bab (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.83617940400000001, 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.83620018500000004, 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:04.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3.c41fdeeb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.83614608899999998, 2024-09-17T21:48:04.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.83621558399999996, 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3.7f293bab (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.82905602499999997, 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.82916678200000005, 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:04.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3.c41fdeeb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.82902234500000005, 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.82916738899999998, 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:04.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:04.516 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 4 2024-09-17T21:48:04.516 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:04.517 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:04.558 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:04.555+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:04.651 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:04.648+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:04.667 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:04.668 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:04.668 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:04.668 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:04.668 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:04.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:04.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:04.763 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:04.760+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7462522289999995, 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:04.833 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:04.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:04.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7451099409999999, 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.840 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:04.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7450938960000002, 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.850 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7450773509999999, 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:04.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7450625859999995, 2024-09-17T21:48:04.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:04.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:04.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7450352809999998, 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.862 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:04.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:04.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7449983930000004, 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:04.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7449596950000004, 2024-09-17T21:48:04.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:04.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:04.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:04.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7449215220000003, 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:04.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:04.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7448840929999996, 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:04.883 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:04.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:04.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:04.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7448270700000004, 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:04.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7447964000000002, 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:04.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:04.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7446520269999999, 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7446331820000003, 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:04.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7446065859999997, 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:04.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:04.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.7445717470000002, 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:04.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.17006878, 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.925 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.170035465, 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:04.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.162945401, 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:04.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:04.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.162911721, 2024-09-17T21:48:04.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:04.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:04.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:04.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:04.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:04.944 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:04.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:04.946 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:04.946 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:04.947 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:04.947 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:04.947 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:04.948 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:04.949 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:04.949 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:04.950 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:04.950 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:04.951 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:04.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:04.951 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:04.942+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:05.046 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:05.043+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:05.092 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.093 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:05.146 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:05.144+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:05.260 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:05.257+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:05.324 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.324 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:05.373 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:05.370+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:05.449 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:05.446+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:05.452 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.452 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:05.550 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:05.547+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:05.568 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.569 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:05.651 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:05.647+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:05.677 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:48:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.697 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.707 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:05.752 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:05.748+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:05.834 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:05.898 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:05.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:05.900 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:05.896+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:06.001 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:05.997+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:06.007 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.007 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:06.102 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:06.099+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:06.142 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.142 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:06.199 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 4] out_osds: [2, 0] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:48:06.199 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:06.199 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-17T21:48:06.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-17T21:48:06.243 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:06.240+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:06.278 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.278 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:06.331 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:06.328+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:06.405 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:06.401+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:06.446 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.447 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:06.541 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:06.538+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:06.575 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:48:06.572+0000 7f0c28614640 -1 mon.a@0(leader).osd e127 definitely_dead 0 2024-09-17T21:48:06.606 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:06.603+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:06.623 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.623 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.634 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:06.670 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:48:06.673 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.0. 2024-09-17T21:48:06.691 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-17T21:48:06.695 INFO:teuthology.orchestra.run.smithi165.stdout:4290 trunc from 0x2c09600 to 0x45e0e00 2024-09-17T21:48:06.732 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:06.729+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.781 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:06.831 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:06.828+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:06.909 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:06.905+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:06.938 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:06.938 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:06.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:06.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:06.967 INFO:teuthology.orchestra.run.smithi165.stdout:4292 write 0x5eff600 thru 0x5f0926c (0x9c6d bytes) 2024-09-17T21:48:06.974 INFO:teuthology.orchestra.run.smithi165.stdout:4293 read 0x25e2200 thru 0x25e375c (0x155d bytes) 2024-09-17T21:48:06.980 INFO:teuthology.orchestra.run.smithi165.stdout:4295 trunc from 0x5f09400 to 0xa906a00 2024-09-17T21:48:07.031 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:07.028+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:07.097 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:07.098 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:07.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:07.112 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:07.112 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:07.109+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:07.130 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:07.146 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:07.147 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:07.164 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:07.164 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:07.181 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:07.181 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:07.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:07.186 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:07.221 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:07.221 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:07.221 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:07.221 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:07.218+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:07.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:07.314 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:07.311+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:07.377 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:07.378 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:07.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:07.413 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:07.411+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:07.514 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:07.511+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:07.535 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.659780595000001, 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:07.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:07.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:07.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623334284999999, 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:07.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623312984000002, 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:07.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623292223, 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:07.551 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:07.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:07.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623271117000002, 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.556 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:07.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:07.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623250628000001, 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:07.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623226531, 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.569 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623204038000001, 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:07.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623176431000001, 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:07.574 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:07.578 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 20.623151590999999, 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.579 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:07.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.471197816, 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.584 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:07.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:07.588 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.471174053999999, 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:07.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.471131665000001, 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:07.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:07.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415383011999999, 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:07.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:07.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.602 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415359917, 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:07.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:07.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415333536999999, 2024-09-17T21:48:07.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:07.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:07.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415316901000001, 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:07.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:07.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:07.616 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415286937000001, 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415274012000001, 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:07.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 16.415263791000001, 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:07.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:07.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:07.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:07.630 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:07.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:07.632 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:07.628+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:07.732 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:07.730+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:07.752 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:07.752 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:07.752 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:07.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:07.833 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:07.830+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:07.909 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:07.909 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:07.909 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:07.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:07.961 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:07.959+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:08.026 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.28378075800001, 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:08.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 102.282736298, 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.033 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:08.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.081550966999998, 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:08.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:08.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8656332990000006, 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:08.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:08.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6631965569999991, 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:08.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:08.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6631393770000003, 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:08.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:08.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:08.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:08.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:08.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6625687960000004, 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:08.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6625408620000002, 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:08.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:08.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6624065100000003, 2024-09-17T21:48:08.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:08.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:08.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6623849750000002, 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:08.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6622240099999992, 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:08.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:08.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6621682450000002, 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:08.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6608216379999998, 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:08.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:08.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:08.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6607700819999991, 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:08.100 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:08.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:08.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6600818210000003, 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.106 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:08.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.66006477, 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:08.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.65912653, 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:08.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:08.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:08.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6590961899999996, 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:08.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:08.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.126 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3646987419999999, 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:08.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:08.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.4635810419999999, 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:08.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:08.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.138 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:08.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:08.140 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:08.136+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:08.241 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:08.238+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:08.288 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:08.288 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:08.288 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:08.296 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:08.342 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:08.339+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:08.402 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:08.402 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:08.403 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:08.403 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:08.411 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:08.442 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:08.440+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544794721, 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544772228, 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544747968, 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544722085, 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54469708400001, 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544657285, 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54464154, 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544630005, 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.544619705, 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.542648462, 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54263531399999, 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54262470800001, 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54261437, 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.542604363, 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.542594412, 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.542579104, 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.540603217, 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.540592261, 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.540579772, 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:08.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.54056930900001, 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:08.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:08.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:08.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:08.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:08.612 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:08.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:08.614 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:08.610+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:08.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:48:08.703 INFO:teuthology.orchestra.run.smithi165.stdout:4297 read 0xb4d000 thru 0xb4f86b (0x286c bytes) 2024-09-17T21:48:08.704 INFO:teuthology.orchestra.run.smithi165.stdout:4298 read 0x4eae600 thru 0x4ebb3fa (0xcdfb bytes) 2024-09-17T21:48:08.704 INFO:teuthology.orchestra.run.smithi165.stdout:4299 write 0xd69400 thru 0xd787ee (0xf3ef bytes) 2024-09-17T21:48:08.704 INFO:teuthology.orchestra.run.smithi165.stdout:4301 write 0x3a5d200 thru 0x3a6c5b5 (0xf3b6 bytes) 2024-09-17T21:48:08.725 INFO:teuthology.orchestra.run.smithi165.stdout:4302 trunc from 0xa906a00 to 0x32b400 2024-09-17T21:48:08.734 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:08.731+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:08.754 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:08.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:08.754 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:08.754 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:08.763 INFO:teuthology.orchestra.run.smithi165.stdout:4303 write 0x20e4a00 thru 0x20f0f56 (0xc557 bytes) 2024-09-17T21:48:08.763 INFO:teuthology.orchestra.run.smithi165.stdout:4304 read 0x44a200 thru 0x44ce7b (0x2c7c bytes) 2024-09-17T21:48:08.763 INFO:teuthology.orchestra.run.smithi165.stdout:4305 punch from 0x1192400 to 0x119d200, (0xae00 bytes) 2024-09-17T21:48:08.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:08.785 INFO:teuthology.orchestra.run.smithi165.stdout:4306 write 0xa9af800 thru 0xa9b2ca8 (0x34a9 bytes) 2024-09-17T21:48:08.786 INFO:teuthology.orchestra.run.smithi165.stdout:4308 punch from 0x7574c00 to 0x757b400, (0x6800 bytes) 2024-09-17T21:48:08.788 INFO:teuthology.orchestra.run.smithi165.stdout:4309 read 0x9887200 thru 0x9896148 (0xef49 bytes) 2024-09-17T21:48:08.849 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:08.846+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:08.916 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:08.913+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:08.929 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:08.930 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:08.930 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:08.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:09.017 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:09.014+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:09.052 INFO:teuthology.orchestra.run.smithi165.stdout:4313 write 0xca3c400 thru 0xca44157 (0x7d58 bytes) 2024-09-17T21:48:09.052 INFO:teuthology.orchestra.run.smithi165.stdout:4314 write 0x8c58c00 thru 0x8c619f8 (0x8df9 bytes) 2024-09-17T21:48:09.052 INFO:teuthology.orchestra.run.smithi165.stdout:4315 punch from 0x3668800 to 0x3675400, (0xcc00 bytes) 2024-09-17T21:48:09.053 INFO:teuthology.orchestra.run.smithi165.stdout:4316 write 0xa799800 thru 0xa7a7bcf (0xe3d0 bytes) 2024-09-17T21:48:09.053 INFO:teuthology.orchestra.run.smithi165.stdout:4317 read 0x2e8e000 thru 0x2e92d3f (0x4d40 bytes) 2024-09-17T21:48:09.053 INFO:teuthology.orchestra.run.smithi165.stdout:4319 read 0x2496000 thru 0x24a2a8c (0xca8d bytes) 2024-09-17T21:48:09.053 INFO:teuthology.orchestra.run.smithi165.stdout:4320 read 0x9eca00 thru 0x9f0ad0 (0x40d1 bytes) 2024-09-17T21:48:09.053 INFO:teuthology.orchestra.run.smithi165.stdout:4323 punch from 0xa17b400 to 0xa185000, (0x9c00 bytes) 2024-09-17T21:48:09.054 INFO:teuthology.orchestra.run.smithi165.stdout:4326 flatten 2024-09-17T21:48:09.085 INFO:teuthology.orchestra.run.smithi165.stdout:4330 write 0x1631800 thru 0x163511e (0x391f bytes) 2024-09-17T21:48:09.085 INFO:teuthology.orchestra.run.smithi165.stdout:4331 read 0x52b7e00 thru 0x52c1996 (0x9b97 bytes) 2024-09-17T21:48:09.088 INFO:teuthology.orchestra.run.smithi165.stdout:4332 trunc from 0xca44200 to 0xafcce00 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.41861192, 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:09.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:09.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.386336945, 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:09.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.38609817699999, 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.218592047000001, 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:09.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:09.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:09.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.211690275999999, 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:09.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:09.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.211574185000003, 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:09.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:09.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014570665000001, 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:09.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014531869999999, 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:09.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014517586, 2024-09-17T21:48:09.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:09.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.01427262, 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.153 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:09.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014190171999999, 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.158 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:09.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:09.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014158362, 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:09.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.014134642999998, 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:09.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:09.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:09.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.627495245999999, 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:09.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:09.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.178 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.617566266000001, 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:09.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:09.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4252315520000001, 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.184 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:09.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:09.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:09.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4249741309999999, 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:09.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.4249412850000001, 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.197 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:09.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:09.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.42492434, 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:09.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:09.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:09.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 1.424882843, 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:09.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:09.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.215 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:09.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:09.216 INFO:teuthology.orchestra.run.smithi165.stdout:4334 write 0x6066400 thru 0x606d4ed (0x70ee bytes) 2024-09-17T21:48:09.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:09.218 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:09.213+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:09.312 INFO:teuthology.orchestra.run.smithi165.stdout:4337 read 0x27de800 thru 0x27edd77 (0xf578 bytes) 2024-09-17T21:48:09.312 INFO:teuthology.orchestra.run.smithi165.stdout:4339 punch from 0x9d22200 to 0x9d23400, (0x1200 bytes) 2024-09-17T21:48:09.316 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:09.313+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:09.333 INFO:teuthology.orchestra.run.smithi165.stdout:4340 write 0xd069c00 thru 0xd06f259 (0x565a bytes) 2024-09-17T21:48:09.333 INFO:teuthology.orchestra.run.smithi165.stdout:4341 punch from 0xa765200 to 0xa765400, (0x200 bytes) 2024-09-17T21:48:09.335 INFO:teuthology.orchestra.run.smithi165.stdout:4342 trunc from 0xd06f400 to 0xc0b5400 2024-09-17T21:48:09.349 INFO:teuthology.orchestra.run.smithi165.stdout:4345 read 0x9732800 thru 0x973ec51 (0xc452 bytes) 2024-09-17T21:48:09.351 INFO:teuthology.orchestra.run.smithi165.stdout:4346 read 0x74c4e00 thru 0x74d1131 (0xc332 bytes) 2024-09-17T21:48:09.366 INFO:teuthology.orchestra.run.smithi165.stdout:4347 trunc from 0xc0b5400 to 0x3e0a400 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:993 3.6 3:67fe6b3d:::rbd_data.1112cb660a00.0000000000000000:head [read 12288~4096 out=4096b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e130)", 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:09.368154+0000", 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.00019541299999999999, 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022124800000000001, 2024-09-17T21:48:09.370 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 993 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368154+0000", 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368154+0000", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368154+0000", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1200000000000002e-07 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368158+0000", 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8160000000000004e-06 2024-09-17T21:48:09.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368167+0000", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0880000000000001e-06 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368172+0000", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9119999999999997e-06 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368214+0000", 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2157000000000003e-05 2024-09-17T21:48:09.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:09.368262+0000", 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8071000000000001e-05 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:48:09.374 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:09.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:09.386 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:09.387 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:09.388 INFO:teuthology.orchestra.run.smithi165.stdout:4348 read 0x1f9bc00 thru 0x1fa34d5 (0x78d6 bytes) 2024-09-17T21:48:09.404 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:09.405 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:09.407 INFO:teuthology.orchestra.run.smithi165.stdout:4349 trunc from 0x3e0a400 to 0xe008400 2024-09-17T21:48:09.417 INFO:teuthology.orchestra.run.smithi165.stdout:4351 punch from 0xc29fc00 to 0xc2ada00, (0xde00 bytes) 2024-09-17T21:48:09.417 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:09.414+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:09.422 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:09.422 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:09.429 INFO:teuthology.orchestra.run.smithi165.stdout:4352 read 0x793ce00 thru 0x793f50c (0x270d bytes) 2024-09-17T21:48:09.435 INFO:teuthology.orchestra.run.smithi165.stdout:4353 trunc from 0xe008400 to 0x8eb6400 2024-09-17T21:48:09.440 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:09.440 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:09.450 INFO:teuthology.orchestra.run.smithi165.stdout:4354 write 0x75bba00 thru 0x75c54c1 (0x9ac2 bytes) 2024-09-17T21:48:09.450 INFO:teuthology.orchestra.run.smithi165.stdout:4355 punch from 0x12f0000 to 0x12f6600, (0x6600 bytes) 2024-09-17T21:48:09.454 INFO:teuthology.orchestra.run.smithi165.stdout:4357 read 0x3d23400 thru 0x3d287ee (0x53ef bytes) 2024-09-17T21:48:09.458 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:09.458 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:09.461 INFO:teuthology.orchestra.run.smithi165.stdout:4358 trunc from 0x8eb6400 to 0x3807000 2024-09-17T21:48:09.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:09.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:09.486 INFO:teuthology.orchestra.run.smithi165.stdout:4359 trunc from 0x3807000 to 0x7188a00 2024-09-17T21:48:09.500 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:09.501 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:09.508 INFO:teuthology.orchestra.run.smithi165.stdout:4361 punch from 0x1235600 to 0x1244000, (0xea00 bytes) 2024-09-17T21:48:09.508 INFO:teuthology.orchestra.run.smithi165.stdout:4367 write 0x3fda800 thru 0x3fe3097 (0x8898 bytes) 2024-09-17T21:48:09.511 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:09.518 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:09.516+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:09.532 INFO:teuthology.orchestra.run.smithi165.stdout:4368 write 0xb7cf800 thru 0xb7dc611 (0xce12 bytes) 2024-09-17T21:48:09.532 INFO:teuthology.orchestra.run.smithi165.stdout:4370 read 0x3f57000 thru 0x3f63c86 (0xcc87 bytes) 2024-09-17T21:48:09.533 INFO:teuthology.orchestra.run.smithi165.stdout:4371 write 0x6f96600 thru 0x6f98af5 (0x24f6 bytes) 2024-09-17T21:48:09.533 INFO:teuthology.orchestra.run.smithi165.stdout:4372 read 0x6e05e00 thru 0x6e11d5e (0xbf5f bytes) 2024-09-17T21:48:09.533 INFO:teuthology.orchestra.run.smithi165.stdout:4374 read 0x4174000 thru 0x417613b (0x213c bytes) 2024-09-17T21:48:09.543 INFO:teuthology.orchestra.run.smithi165.stdout:4376 trunc from 0xb7dc800 to 0x2fcc600 2024-09-17T21:48:09.569 INFO:teuthology.orchestra.run.smithi165.stdout:4379 punch from 0x238fc00 to 0x239dc00, (0xe000 bytes) 2024-09-17T21:48:09.572 INFO:teuthology.orchestra.run.smithi165.stdout:4380 trunc from 0x2fcc600 to 0x5ffca00 2024-09-17T21:48:09.618 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:09.615+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:09.661 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:09.661 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:09.670 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:48:09.744 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:09.741+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:09.797 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:09.797 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:09.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:09.837 INFO:teuthology.orchestra.run.smithi165.stdout:4381 trunc from 0x5ffca00 to 0x3df8600 2024-09-17T21:48:09.846 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:09.843+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:09.849 INFO:teuthology.orchestra.run.smithi165.stdout:4382 read 0x67200 thru 0x6d250 (0x6051 bytes) 2024-09-17T21:48:09.920 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:09.918+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:09.953 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:09.953 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:09.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:10.021 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:10.019+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:10.069 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.069 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.079 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:10.122 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:10.119+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.191 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.200 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:10.243 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:10.241+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:10.306 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.306 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:10.323 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:10.321+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:10.424 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:10.421+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:10.462 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.462 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:10.525 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:10.523+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:10.577 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.577 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:10.626 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:10.623+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:10.696 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.696 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.697 INFO:teuthology.orchestra.run.smithi165.stdout:4384 punch from 0x908400 to 0x90da00, (0x5600 bytes) 2024-09-17T21:48:10.708 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:10.723 INFO:teuthology.orchestra.run.smithi165.stdout:4386 write 0xd0d0000 thru 0xd0d5175 (0x5176 bytes) 2024-09-17T21:48:10.725 INFO:teuthology.orchestra.run.smithi165.stdout:4387 write 0x966aa00 thru 0x967811f (0xd720 bytes) 2024-09-17T21:48:10.725 INFO:teuthology.orchestra.run.smithi165.stdout:4388 read 0x8a58400 thru 0x8a64b4c (0xc74d bytes) 2024-09-17T21:48:10.725 INFO:teuthology.orchestra.run.smithi165.stdout:4389 write 0x8f80400 thru 0x8f87d90 (0x7991 bytes) 2024-09-17T21:48:10.727 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:10.724+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:10.739 INFO:teuthology.orchestra.run.smithi165.stdout:4390 trunc from 0xd0d5200 to 0x3837800 2024-09-17T21:48:10.785 INFO:teuthology.orchestra.run.smithi165.stdout:4391 write 0xca5b400 thru 0xca5e10c (0x2d0d bytes) 2024-09-17T21:48:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:4393 read 0x4b01c00 thru 0x4b05879 (0x3c7a bytes) 2024-09-17T21:48:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:4395 read 0x755f600 thru 0x756394e (0x434f bytes) 2024-09-17T21:48:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:4396 write 0x8a69e00 thru 0x8a71122 (0x7323 bytes) 2024-09-17T21:48:10.786 INFO:teuthology.orchestra.run.smithi165.stdout:4399 punch from 0xa67a00 to 0xa6a800, (0x2e00 bytes) 2024-09-17T21:48:10.787 INFO:teuthology.orchestra.run.smithi165.stdout:4401 trunc from 0xca5e200 to 0xc61b600 2024-09-17T21:48:10.805 INFO:teuthology.orchestra.run.smithi165.stdout:4403 punch from 0x8df4800 to 0x8e01e00, (0xd600 bytes) 2024-09-17T21:48:10.809 INFO:teuthology.orchestra.run.smithi165.stdout:4404 read 0x9b34400 thru 0x9b41133 (0xcd34 bytes) 2024-09-17T21:48:10.809 INFO:teuthology.orchestra.run.smithi165.stdout:4405 read 0x9beb800 thru 0x9bfaf8e (0xf78f bytes) 2024-09-17T21:48:10.809 INFO:teuthology.orchestra.run.smithi165.stdout:4406 read 0x3e2f000 thru 0x3e399e9 (0xa9ea bytes) 2024-09-17T21:48:10.809 INFO:teuthology.orchestra.run.smithi165.stdout:4407 write 0x7525e00 thru 0x7529ba7 (0x3da8 bytes) 2024-09-17T21:48:10.809 INFO:teuthology.orchestra.run.smithi165.stdout:4410 write 0x272e400 thru 0x2732e2d (0x4a2e bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4411 read 0xa63fe00 thru 0xa64a577 (0xa778 bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4412 read 0x3bff000 thru 0x3c0c74e (0xd74f bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4413 punch from 0x64d0a00 to 0x64da600, (0x9c00 bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4414 write 0x908dc00 thru 0x9093e9c (0x629d bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4415 read 0x8a20e00 thru 0x8a231a7 (0x23a8 bytes) 2024-09-17T21:48:10.810 INFO:teuthology.orchestra.run.smithi165.stdout:4416 clone 39 order 21 su 262144 sc 10 2024-09-17T21:48:10.828 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:10.824+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:10.832 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.832 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:10.928 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:10.925+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:10.952 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:10.952 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:10.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:11.029 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:11.027+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:11.108 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:11.108 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:11.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:11.161 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:11.158+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:11.225 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:11.225 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:11.232 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:11.230+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:11.234 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:11.334 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:11.331+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:11.382 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:11.382 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:11.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:11.434 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:11.431+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:11.463 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:11.498 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:11.498 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:11.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:11.548 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:11.545+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:11.586 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:11.604 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:11.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:11.622 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:11.622 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:11.636 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:11.634+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:11.640 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:11.640 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:11.657 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:11.657 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:11.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.571821613000001, 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:11.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:11.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.663 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570679325, 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:11.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.57066328, 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:11.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570646735, 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:11.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:11.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570631970000001, 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:11.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:11.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:11.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:11.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570604664999999, 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:11.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:11.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570567777000001, 2024-09-17T21:48:11.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:11.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:11.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570529079, 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.692 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:11.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570490906, 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.697 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:11.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:11.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:11.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:11.701 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570453477000001, 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:11.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570396454000001, 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:11.711 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570365784, 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:11.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.716 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570221411, 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.721 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570202566000001, 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:11.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570175969999999, 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.570141131, 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:11.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:11.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.9956381639999998, 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:11.737 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:11.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:11.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.9956048490000002, 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:11.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:11.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:11.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:11.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.9885147849999996, 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:11.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:11.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 7.988481105, 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:11.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:11.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:11.759 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:11.760 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 4, 0] out_osds: [2] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:48:11.760 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:11.760 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T21:48:11.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T21:48:11.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:11.761 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:11.762 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:11.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:11.763 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:11.759+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:11.888 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:11.885+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:11.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-17T21:48:11.965 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:11.962+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:11.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:11.966 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:11.967 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:11.967 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:11.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:12.065 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:12.062+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:12.081 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:12.085 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:12.085 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-09-17T21:48:12.086 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T21:48:12.117 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-17T21:48:12.130 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:12.130 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:12.130 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:12.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:12.184 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:12.181+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:12.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T21:48:12.263 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:12.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 97.387180580999996, 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:12.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:12.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350734271, 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:12.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:12.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.35071297, 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:12.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350692209000002, 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.279 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:12.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.283 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350671103, 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:12.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350650613999999, 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:12.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350626516999998, 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350604023999999, 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:12.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350576416999999, 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:12.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.350551577000001, 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.307 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.311 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.198597801999998, 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:12.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.19857404, 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.317 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.198531651, 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:12.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:12.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:12.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142782998000001, 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:12.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:12.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142759903000002, 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.334 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142733523, 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:12.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:12.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:12.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142716886999999, 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.339 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:12.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:12.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142686922999999, 2024-09-17T21:48:12.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:12.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142673997999999, 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.352 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.142663776999999, 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:12.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:12.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:12.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:12.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:12.360 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:12.355+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4417 write 0x2281800 thru 0x22888d1 (0x70d2 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4418 read 0x334b200 thru 0x334d41a (0x221b bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4419 punch from 0x7bb1200 to 0x7bb8000, (0x6e00 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4420 read 0x4a85200 thru 0x4a914d3 (0xc2d4 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4425 punch from 0x329d800 to 0x32ab800, (0xe000 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4428 write 0xa462a00 thru 0xa462bca (0x1cb bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4435 read 0x260800 thru 0x2649b4 (0x41b5 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4437 punch from 0x14cd200 to 0x14cee00, (0x1c00 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4438 read 0x7a5d200 thru 0x7a60c58 (0x3a59 bytes) 2024-09-17T21:48:12.361 INFO:teuthology.orchestra.run.smithi165.stdout:4441 write 0xb892400 thru 0xb8a206d (0xfc6e bytes) 2024-09-17T21:48:12.362 INFO:teuthology.orchestra.run.smithi165.stdout:4445 write 0xa190e00 thru 0xa198cb6 (0x7eb7 bytes) 2024-09-17T21:48:12.362 INFO:teuthology.orchestra.run.smithi165.stdout:4447 write 0xd326400 thru 0xd328958 (0x2559 bytes) 2024-09-17T21:48:12.362 INFO:teuthology.orchestra.run.smithi165.stdout:4449 write 0x817dc00 thru 0x818a543 (0xc944 bytes) 2024-09-17T21:48:12.362 INFO:teuthology.orchestra.run.smithi165.stdout:4450 punch from 0x210ac00 to 0x2117400, (0xc800 bytes) 2024-09-17T21:48:12.362 INFO:teuthology.orchestra.run.smithi165.stdout:4451 trunc from 0xd328a00 to 0xeabc400 2024-09-17T21:48:12.364 INFO:teuthology.orchestra.run.smithi165.stdout:4453 write 0xb3f1c00 thru 0xb3fd62e (0xba2f bytes) 2024-09-17T21:48:12.365 INFO:teuthology.orchestra.run.smithi165.stdout:4454 read 0xabdf400 thru 0xabede6b (0xea6c bytes) 2024-09-17T21:48:12.378 INFO:teuthology.orchestra.run.smithi165.stdout:4455 trunc from 0xeabc400 to 0x3a5ba00 2024-09-17T21:48:12.406 INFO:teuthology.orchestra.run.smithi165.stdout:4458 read 0x2ebd000 thru 0x2ec60d8 (0x90d9 bytes) 2024-09-17T21:48:12.411 INFO:teuthology.orchestra.run.smithi165.stdout:4460 trunc from 0x3a5ba00 to 0xaa0400 2024-09-17T21:48:12.459 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:12.456+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:12.463 INFO:teuthology.orchestra.run.smithi165.stdout:4461 write 0xd72a600 thru 0xd73a284 (0xfc85 bytes) 2024-09-17T21:48:12.464 INFO:teuthology.orchestra.run.smithi165.stdout:4463 write 0x29c7600 thru 0x29ca047 (0x2a48 bytes) 2024-09-17T21:48:12.464 INFO:teuthology.orchestra.run.smithi165.stdout:4465 punch from 0x6b19a00 to 0x6b23a00, (0xa000 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4467 write 0xccbfa00 thru 0xcccf5e3 (0xfbe4 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4468 write 0x31c3a00 thru 0x31c8d03 (0x5304 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4470 write 0x94d4a00 thru 0x94d52f4 (0x8f5 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4471 write 0x48b3000 thru 0x48bcbb1 (0x9bb2 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4472 read 0x7874c00 thru 0x78846d2 (0xfad3 bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4474 read 0x8496600 thru 0x849bfcd (0x59ce bytes) 2024-09-17T21:48:12.466 INFO:teuthology.orchestra.run.smithi165.stdout:4476 punch from 0xbe6e000 to 0xbe74a00, (0x6a00 bytes) 2024-09-17T21:48:12.467 INFO:teuthology.orchestra.run.smithi165.stdout:4479 read 0x6684a00 thru 0x6692b5d (0xe15e bytes) 2024-09-17T21:48:12.475 INFO:teuthology.orchestra.run.smithi165.stdout:4480 trunc from 0xd73a400 to 0x690a000 2024-09-17T21:48:12.489 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:12.490 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:12.490 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:12.490 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:12.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:12.504 INFO:teuthology.orchestra.run.smithi165.stdout:4483 write 0xad0bc00 thru 0xad12075 (0x6476 bytes) 2024-09-17T21:48:12.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T21:48:12.512 INFO:teuthology.orchestra.run.smithi165.stdout:4484 write 0x53b6a00 thru 0x53b8ff6 (0x25f7 bytes) 2024-09-17T21:48:12.513 INFO:teuthology.orchestra.run.smithi165.stdout:4486 punch from 0x3509a00 to 0x350c800, (0x2e00 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4488 read 0x9ddfa00 thru 0x9de98b4 (0x9eb5 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4489 read 0x7b27600 thru 0x7b276b0 (0xb1 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4490 write 0x2b26c00 thru 0x2b2ea56 (0x7e57 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4491 punch from 0x41bc00 to 0x429400, (0xd800 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4493 write 0x15f9200 thru 0x1601788 (0x8589 bytes) 2024-09-17T21:48:12.514 INFO:teuthology.orchestra.run.smithi165.stdout:4494 read 0x719600 thru 0x723926 (0xa327 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4495 punch from 0xa9f2600 to 0xa9fdc00, (0xb600 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4496 write 0x2723200 thru 0x272f1f8 (0xbff9 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4497 write 0x2130400 thru 0x213c99a (0xc59b bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4500 punch from 0x336a000 to 0x336e600, (0x4600 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4502 punch from 0x1554c00 to 0x155ba00, (0x6e00 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4503 read 0x1843600 thru 0x184a791 (0x7192 bytes) 2024-09-17T21:48:12.516 INFO:teuthology.orchestra.run.smithi165.stdout:4504 trunc from 0xad12200 to 0xa183e00 2024-09-17T21:48:12.540 INFO:teuthology.orchestra.run.smithi165.stdout:4505 trunc from 0xa183e00 to 0x4fe8c00 2024-09-17T21:48:12.554 INFO:teuthology.orchestra.run.smithi165.stdout:4506 punch from 0x3bd7400 to 0x3be2a00, (0xb600 bytes) 2024-09-17T21:48:12.557 INFO:teuthology.orchestra.run.smithi165.stdout:4507 write 0xb51c00 thru 0xb56f51 (0x5352 bytes) 2024-09-17T21:48:12.559 INFO:teuthology.orchestra.run.smithi165.stdout:4508 trunc from 0x4fe8c00 to 0x6933400 2024-09-17T21:48:12.559 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:12.556+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:12.571 INFO:teuthology.orchestra.run.smithi165.stdout:4509 write 0x39a2a00 thru 0x39acb88 (0xa189 bytes) 2024-09-17T21:48:12.595 INFO:teuthology.orchestra.run.smithi165.stdout:4511 write 0xc030e00 thru 0xc040df9 (0xfffa bytes) 2024-09-17T21:48:12.608 INFO:teuthology.orchestra.run.smithi165.stdout:4512 trunc from 0xc040e00 to 0x21f7000 2024-09-17T21:48:12.625 INFO:teuthology.orchestra.run.smithi165.stdout:4514 read 0x482200 thru 0x489f1b (0x7d1c bytes) 2024-09-17T21:48:12.627 INFO:teuthology.orchestra.run.smithi165.stdout:4516 read 0x19e9200 thru 0x19f2962 (0x9763 bytes) 2024-09-17T21:48:12.627 INFO:teuthology.orchestra.run.smithi165.stdout:4519 read 0x61ba00 thru 0x622f54 (0x7555 bytes) 2024-09-17T21:48:12.639 INFO:teuthology.orchestra.run.smithi165.stdout:4522 trunc from 0x21f7000 to 0xc54ca00 2024-09-17T21:48:12.648 INFO:teuthology.orchestra.run.smithi165.stdout:4525 write 0x90a6600 thru 0x90addd7 (0x77d8 bytes) 2024-09-17T21:48:12.658 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:12.658 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:12.658 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:12.660 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:12.657+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:12.662 INFO:teuthology.orchestra.run.smithi165.stdout:4526 trunc from 0xc54ca00 to 0x1dc5c00 2024-09-17T21:48:12.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:12.680 INFO:teuthology.orchestra.run.smithi165.stdout:4528 read 0x1a5c600 thru 0x1a5fd3f (0x3740 bytes) 2024-09-17T21:48:12.682 INFO:teuthology.orchestra.run.smithi165.stdout:4529 punch from 0x1c15c00 to 0x1c1c800, (0x6c00 bytes) 2024-09-17T21:48:12.683 INFO:teuthology.orchestra.run.smithi165.stdout:4530 punch from 0x1a50600 to 0x1a5f200, (0xec00 bytes) 2024-09-17T21:48:12.685 INFO:teuthology.orchestra.run.smithi165.stdout:4531 write 0xfba000 thru 0xfc8c47 (0xec48 bytes) 2024-09-17T21:48:12.685 INFO:teuthology.orchestra.run.smithi165.stdout:4534 punch from 0xdbe000 to 0xdcd400, (0xf400 bytes) 2024-09-17T21:48:12.691 INFO:teuthology.orchestra.run.smithi165.stdout:4537 trunc from 0x1dc5c00 to 0x8528e00 2024-09-17T21:48:12.699 INFO:teuthology.orchestra.run.smithi165.stdout:4539 read 0x81e9800 thru 0x81f8b4a (0xf34b bytes) 2024-09-17T21:48:12.700 INFO:teuthology.orchestra.run.smithi165.stdout:4540 read 0x3ae9200 thru 0x3af61bb (0xcfbc bytes) 2024-09-17T21:48:12.701 INFO:teuthology.orchestra.run.smithi165.stdout:4544 punch from 0x6c9bc00 to 0x6c9e800, (0x2c00 bytes) 2024-09-17T21:48:12.702 INFO:teuthology.orchestra.run.smithi165.stdout:4546 read 0x1496200 thru 0x14a133f (0xb140 bytes) 2024-09-17T21:48:12.702 INFO:teuthology.orchestra.run.smithi165.stdout:4548 write 0x54bce00 thru 0x54c84b9 (0xb6ba bytes) 2024-09-17T21:48:12.703 INFO:teuthology.orchestra.run.smithi165.stdout:4549 punch from 0x6cce200 to 0x6cda600, (0xc400 bytes) 2024-09-17T21:48:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:48:12.717 INFO:teuthology.orchestra.run.smithi165.stdout:4552 write 0xe0e3800 thru 0xe0e5755 (0x1f56 bytes) 2024-09-17T21:48:12.719 INFO:teuthology.orchestra.run.smithi165.stdout:4555 read 0xad1ea00 thru 0xad21ba4 (0x31a5 bytes) 2024-09-17T21:48:12.719 INFO:teuthology.orchestra.run.smithi165.stdout:4556 punch from 0x2594200 to 0x2597e00, (0x3c00 bytes) 2024-09-17T21:48:12.720 INFO:teuthology.orchestra.run.smithi165.stdout:4557 write 0xbb33e00 thru 0xbb3f33b (0xb53c bytes) 2024-09-17T21:48:12.725 INFO:teuthology.orchestra.run.smithi165.stdout:4558 trunc from 0xe0e5800 to 0x97d4600 2024-09-17T21:48:12.740 INFO:teuthology.orchestra.run.smithi165.stdout:4560 write 0x8c29e00 thru 0x8c331a8 (0x93a9 bytes) 2024-09-17T21:48:12.744 INFO:teuthology.orchestra.run.smithi165.stdout:4561 trunc from 0x97d4600 to 0xb817a00 2024-09-17T21:48:12.756 INFO:teuthology.orchestra.run.smithi165.stdout:4562 punch from 0x9410e00 to 0x9418000, (0x7200 bytes) 2024-09-17T21:48:12.759 INFO:teuthology.orchestra.run.smithi165.stdout:4563 read 0x7a25200 thru 0x7a2613f (0xf40 bytes) 2024-09-17T21:48:12.759 INFO:teuthology.orchestra.run.smithi165.stdout:4564 write 0xa34a000 thru 0xa34e065 (0x4066 bytes) 2024-09-17T21:48:12.761 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:12.758+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:12.772 INFO:teuthology.orchestra.run.smithi165.stdout:4565 write 0xc94ce00 thru 0xc955d70 (0x8f71 bytes) 2024-09-17T21:48:12.773 INFO:teuthology.orchestra.run.smithi165.stdout:4566 read 0xa258400 thru 0xa264436 (0xc037 bytes) 2024-09-17T21:48:12.773 INFO:teuthology.orchestra.run.smithi165.stdout:4567 write 0xcef200 thru 0xceffba (0xdbb bytes) 2024-09-17T21:48:12.777 INFO:teuthology.orchestra.run.smithi165.stdout:4568 read 0x4842200 thru 0x48475b1 (0x53b2 bytes) 2024-09-17T21:48:12.777 INFO:teuthology.orchestra.run.smithi165.stdout:4569 write 0xc424800 thru 0xc42bdd4 (0x75d5 bytes) 2024-09-17T21:48:12.778 INFO:teuthology.orchestra.run.smithi165.stdout:4573 write 0x7a4a200 thru 0x7a4d863 (0x3664 bytes) 2024-09-17T21:48:12.778 INFO:teuthology.orchestra.run.smithi165.stdout:4574 write 0x2bffe00 thru 0x2c06f85 (0x7186 bytes) 2024-09-17T21:48:12.778 INFO:teuthology.orchestra.run.smithi165.stdout:4579 punch from 0xa81fa00 to 0xa829c00, (0xa200 bytes) 2024-09-17T21:48:12.778 INFO:teuthology.orchestra.run.smithi165.stdout:4580 punch from 0x93bf400 to 0x93c5800, (0x6400 bytes) 2024-09-17T21:48:12.778 INFO:teuthology.orchestra.run.smithi165.stdout:4581 read 0xb0c4c00 thru 0xb0ccc11 (0x8012 bytes) 2024-09-17T21:48:12.779 INFO:teuthology.orchestra.run.smithi165.stdout:4582 read 0x7274c00 thru 0x7275276 (0x677 bytes) 2024-09-17T21:48:12.779 INFO:teuthology.orchestra.run.smithi165.stdout:4585 write 0x6e9f800 thru 0x6ea591a (0x611b bytes) 2024-09-17T21:48:12.779 INFO:teuthology.orchestra.run.smithi165.stdout:4587 punch from 0x85e6000 to 0x85ef600, (0x9600 bytes) 2024-09-17T21:48:12.784 INFO:teuthology.orchestra.run.smithi165.stdout:4589 trunc from 0xc955e00 to 0x7f8b800 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:12.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.048378607, 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:12.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:12.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 107.047334147, 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:12.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.846148816000003, 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.630231148, 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.427794406, 2024-09-17T21:48:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.427737226, 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.427166645, 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.427138711, 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.427004359, 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.426982824, 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.426821859, 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.426766094, 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.425419486999999, 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.425367931, 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.42467967, 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.424662618999999, 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.423724378999999, 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.423694039000001, 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.1292965909999992, 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:12.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:12.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:12.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2281788910000007, 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:12.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:12.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:12.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:12.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:12.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:12.901 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:12.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:12.903 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:12.899+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:12.904 INFO:teuthology.orchestra.run.smithi165.stdout:4591 read 0x5b64000 thru 0x5b6bb4a (0x7b4b bytes) 2024-09-17T21:48:12.904 INFO:teuthology.orchestra.run.smithi165.stdout:4592 write 0x46a7e00 thru 0x46a8859 (0xa5a bytes) 2024-09-17T21:48:12.904 INFO:teuthology.orchestra.run.smithi165.stdout:4593 read 0x5cbe200 thru 0x5cbf3ff (0x1200 bytes) 2024-09-17T21:48:12.904 INFO:teuthology.orchestra.run.smithi165.stdout:4595 trunc from 0x7f8b800 to 0x6407e00 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4598 write 0x807fe00 thru 0x8085638 (0x5839 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4599 read 0xbf4200 thru 0xc03067 (0xee68 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4600 read 0x7dcca00 thru 0x7dd712b (0xa72c bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4601 read 0x53b3800 thru 0x53bfca7 (0xc4a8 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4604 write 0x9bace00 thru 0x9bad329 (0x52a bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4605 read 0x874ec00 thru 0x875de33 (0xf234 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4608 write 0xcf10400 thru 0xcf1a455 (0xa056 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4613 punch from 0x4abd600 to 0x4ac6c00, (0x9600 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4614 trunc from 0xcf1a600 to 0xc4cf400 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4615 punch from 0xb453400 to 0xb45e000, (0xac00 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4617 read 0x3dbc000 thru 0x3dcb0e5 (0xf0e6 bytes) 2024-09-17T21:48:12.905 INFO:teuthology.orchestra.run.smithi165.stdout:4619 punch from 0x4e77400 to 0x4e85800, (0xe400 bytes) 2024-09-17T21:48:12.908 INFO:teuthology.orchestra.run.smithi165.stdout:4620 write 0xe527800 thru 0xe53698d (0xf18e bytes) 2024-09-17T21:48:12.908 INFO:teuthology.orchestra.run.smithi165.stdout:4621 read 0x7c1de00 thru 0x7c20948 (0x2b49 bytes) 2024-09-17T21:48:12.908 INFO:teuthology.orchestra.run.smithi165.stdout:4623 punch from 0xb89f800 to 0xb8ad400, (0xdc00 bytes) 2024-09-17T21:48:12.912 INFO:teuthology.orchestra.run.smithi165.stdout:4626 trunc from 0xe536a00 to 0xc37e600 2024-09-17T21:48:12.925 INFO:teuthology.orchestra.run.smithi165.stdout:4627 read 0x20f0200 thru 0x20f1382 (0x1183 bytes) 2024-09-17T21:48:12.932 INFO:teuthology.orchestra.run.smithi165.stdout:4629 trunc from 0xc37e600 to 0x70e2800 2024-09-17T21:48:12.951 INFO:teuthology.orchestra.run.smithi165.stdout:4632 trunc from 0x70e2800 to 0xcc44a00 2024-09-17T21:48:12.960 INFO:teuthology.orchestra.run.smithi165.stdout:4633 read 0x8e13200 thru 0x8e1804b (0x4e4c bytes) 2024-09-17T21:48:12.961 INFO:teuthology.orchestra.run.smithi165.stdout:4634 read 0x613c00 thru 0x61f0d7 (0xb4d8 bytes) 2024-09-17T21:48:12.962 INFO:teuthology.orchestra.run.smithi165.stdout:4635 trunc from 0xcc44a00 to 0xc9e9c00 2024-09-17T21:48:12.973 INFO:teuthology.orchestra.run.smithi165.stdout:4643 write 0x1c21600 thru 0x1c2fe73 (0xe874 bytes) 2024-09-17T21:48:12.980 INFO:teuthology.orchestra.run.smithi165.stdout:4648 write 0x8af2a00 thru 0x8af8939 (0x5f3a bytes) 2024-09-17T21:48:12.981 INFO:teuthology.orchestra.run.smithi165.stdout:4650 write 0x6685e00 thru 0x6693b26 (0xdd27 bytes) 2024-09-17T21:48:12.981 INFO:teuthology.orchestra.run.smithi165.stdout:4652 read 0x5ad7e00 thru 0x5ae2350 (0xa551 bytes) 2024-09-17T21:48:12.981 INFO:teuthology.orchestra.run.smithi165.stdout:4653 read 0xc106a00 thru 0xc107f90 (0x1591 bytes) 2024-09-17T21:48:12.994 INFO:teuthology.orchestra.run.smithi165.stdout:4654 trunc from 0xc9e9c00 to 0xc83000 2024-09-17T21:48:13.002 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:13.000+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:13.023 INFO:teuthology.orchestra.run.smithi165.stdout:4655 punch from 0x95c600 to 0x965000, (0x8a00 bytes) 2024-09-17T21:48:13.045 INFO:teuthology.orchestra.run.smithi165.stdout:4656 write 0x5acfa00 thru 0x5ad8cba (0x92bb bytes) 2024-09-17T21:48:13.051 INFO:teuthology.orchestra.run.smithi165.stdout:4657 trunc from 0x5ad8e00 to 0xa5ef000 2024-09-17T21:48:13.058 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:13.058 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:13.058 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:13.058 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:13.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:13.069 INFO:teuthology.orchestra.run.smithi165.stdout:4659 trunc from 0xa5ef000 to 0x9780200 2024-09-17T21:48:13.085 INFO:teuthology.orchestra.run.smithi165.stdout:4662 read 0x7f1de00 thru 0x7f1fa71 (0x1c72 bytes) 2024-09-17T21:48:13.095 INFO:teuthology.orchestra.run.smithi165.stdout:4663 trunc from 0x9780200 to 0x78ec00 2024-09-17T21:48:13.108 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:13.106+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:13.115 INFO:teuthology.orchestra.run.smithi165.stdout:4664 read 0x4bfc00 thru 0x4c757e (0x797f bytes) 2024-09-17T21:48:13.115 INFO:teuthology.orchestra.run.smithi165.stdout:4665 read 0x432a00 thru 0x43b27f (0x8880 bytes) 2024-09-17T21:48:13.141 INFO:teuthology.orchestra.run.smithi165.stdout:4668 write 0xe1a6400 thru 0xe1b0af5 (0xa6f6 bytes) 2024-09-17T21:48:13.143 INFO:teuthology.orchestra.run.smithi165.stdout:4669 read 0xa6baa00 thru 0xa6bdaa0 (0x30a1 bytes) 2024-09-17T21:48:13.143 INFO:teuthology.orchestra.run.smithi165.stdout:4671 punch from 0xc493e00 to 0xc49b000, (0x7200 bytes) 2024-09-17T21:48:13.144 INFO:teuthology.orchestra.run.smithi165.stdout:4672 trunc from 0xe1b0c00 to 0xdc17600 2024-09-17T21:48:13.158 INFO:teuthology.orchestra.run.smithi165.stdout:4673 write 0xdb09c00 thru 0xdb0b84b (0x1c4c bytes) 2024-09-17T21:48:13.161 INFO:teuthology.orchestra.run.smithi165.stdout:4675 trunc from 0xdc17600 to 0xbcaa600 2024-09-17T21:48:13.176 INFO:teuthology.orchestra.run.smithi165.stdout:4676 read 0x2ebe200 thru 0x2ec3484 (0x5285 bytes) 2024-09-17T21:48:13.177 INFO:teuthology.orchestra.run.smithi165.stdout:4677 write 0xb420200 thru 0xb42cd3a (0xcb3b bytes) 2024-09-17T21:48:13.177 INFO:teuthology.orchestra.run.smithi165.stdout:4678 trunc from 0xbcaa600 to 0xc04a000 2024-09-17T21:48:13.195 INFO:teuthology.orchestra.run.smithi165.stdout:4679 write 0x9d6cc00 thru 0x9d72778 (0x5b79 bytes) 2024-09-17T21:48:13.204 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:13.201+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:13.212 INFO:teuthology.orchestra.run.smithi165.stdout:4680 write 0xc736200 thru 0xc736bd6 (0x9d7 bytes) 2024-09-17T21:48:13.212 INFO:teuthology.orchestra.run.smithi165.stdout:4681 write 0x78fb000 thru 0x78fd565 (0x2566 bytes) 2024-09-17T21:48:13.220 INFO:teuthology.orchestra.run.smithi165.stdout:4684 trunc from 0xc736c00 to 0x5068c00 2024-09-17T21:48:13.225 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:13.225 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:13.225 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:13.225 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:13.225 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:13.235 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:13.242 INFO:teuthology.orchestra.run.smithi165.stdout:4685 punch from 0x2fb200 to 0x2fca00, (0x1800 bytes) 2024-09-17T21:48:13.263 INFO:teuthology.orchestra.run.smithi165.stdout:4688 write 0xe382e00 thru 0xe387bd2 (0x4dd3 bytes) 2024-09-17T21:48:13.270 INFO:teuthology.orchestra.run.smithi165.stdout:4689 write 0xafa3e00 thru 0xafa4443 (0x644 bytes) 2024-09-17T21:48:13.270 INFO:teuthology.orchestra.run.smithi165.stdout:4692 punch from 0xb740600 to 0xb742e00, (0x2800 bytes) 2024-09-17T21:48:13.271 INFO:teuthology.orchestra.run.smithi165.stdout:4694 read 0x17e0200 thru 0x17e4ece (0x4ccf bytes) 2024-09-17T21:48:13.271 INFO:teuthology.orchestra.run.smithi165.stdout:4696 read 0x84a2200 thru 0x84a90bb (0x6ebc bytes) 2024-09-17T21:48:13.279 INFO:teuthology.orchestra.run.smithi165.stdout:4697 trunc from 0xe387c00 to 0x7b31600 2024-09-17T21:48:13.292 INFO:teuthology.orchestra.run.smithi165.stdout:4698 punch from 0x27f3000 to 0x27f9a00, (0x6a00 bytes) 2024-09-17T21:48:13.300 INFO:teuthology.orchestra.run.smithi165.stdout:4700 punch from 0x35ca800 to 0x35d4e00, (0xa600 bytes) 2024-09-17T21:48:13.300 INFO:teuthology.orchestra.run.smithi165.stdout:4701 read 0x18c1600 thru 0x18cd6da (0xc0db bytes) 2024-09-17T21:48:13.304 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:13.301+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:13.312 INFO:teuthology.orchestra.run.smithi165.stdout:4703 write 0x8373200 thru 0x8374a80 (0x1881 bytes) 2024-09-17T21:48:13.331 INFO:teuthology.orchestra.run.smithi165.stdout:4705 write 0xe842e00 thru 0xe84fffb (0xd1fc bytes) 2024-09-17T21:48:13.332 INFO:teuthology.orchestra.run.smithi165.stdout:4706 punch from 0x3050000 to 0x3056c00, (0x6c00 bytes) 2024-09-17T21:48:13.334 INFO:teuthology.orchestra.run.smithi165.stdout:4707 trunc from 0xe850000 to 0xecfdc00 2024-09-17T21:48:13.345 INFO:teuthology.orchestra.run.smithi165.stdout:4709 write 0xa8ed600 thru 0xa8f7533 (0x9f34 bytes) 2024-09-17T21:48:13.352 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378500184, 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378477691, 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378453431, 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378427548, 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.37840254699999, 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378362748, 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378347003, 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378335468, 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:13.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.378325168, 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:13.390 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376353925, 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376340777, 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.37633017100001, 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376319833, 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:13.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376309826, 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:13.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.416 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376299875, 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:13.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.376284567, 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:13.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.37430868, 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.374297724, 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:13.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:13.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.374285235, 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:13.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:13.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.37427477200001, 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:13.438 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:13.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:13.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:13.442 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:13.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4710 trunc from 0xecfdc00 to 0x7c68200 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4714 punch from 0x6610200 to 0x6616600, (0x6400 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4715 write 0xc9ae400 thru 0xc9b5391 (0x6f92 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4717 write 0x5b2ca00 thru 0x5b2e273 (0x1874 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4720 write 0x612fe00 thru 0x61354c2 (0x56c3 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4721 write 0x4c48600 thru 0x4c53b22 (0xb523 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4722 read 0x3bda600 thru 0x3be14cd (0x6ece bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4723 write 0xb320a00 thru 0xb3272c8 (0x68c9 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4724 trunc from 0xc9b5400 to 0xcede200 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4725 read 0x8e9a400 thru 0x8ea5be0 (0xb7e1 bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4728 write 0x263600 thru 0x268d4e (0x574f bytes) 2024-09-17T21:48:13.446 INFO:teuthology.orchestra.run.smithi165.stdout:4729 trunc from 0xcede200 to 0x6b56e00 2024-09-17T21:48:13.447 INFO:teuthology.orchestra.run.smithi165.stdout:4730 punch from 0x1e49c00 to 0x1e4f800, (0x5c00 bytes) 2024-09-17T21:48:13.447 INFO:teuthology.orchestra.run.smithi165.stdout:4732 punch from 0x1f30200 to 0x1f34600, (0x4400 bytes) 2024-09-17T21:48:13.447 INFO:teuthology.orchestra.run.smithi165.stdout:4734 punch from 0x670f400 to 0x670fe00, (0xa00 bytes) 2024-09-17T21:48:13.447 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:13.440+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:13.454 INFO:teuthology.orchestra.run.smithi165.stdout:4735 write 0x8bea800 thru 0x8bedd23 (0x3524 bytes) 2024-09-17T21:48:13.458 INFO:teuthology.orchestra.run.smithi165.stdout:4736 read 0x1197400 thru 0x11a59fa (0xe5fb bytes) 2024-09-17T21:48:13.458 INFO:teuthology.orchestra.run.smithi165.stdout:4738 trunc from 0x8bede00 to 0x9ff8000 2024-09-17T21:48:13.469 INFO:teuthology.orchestra.run.smithi165.stdout:4739 read 0x2355a00 thru 0x235a17e (0x477f bytes) 2024-09-17T21:48:13.470 INFO:teuthology.orchestra.run.smithi165.stdout:4742 write 0x6d6d600 thru 0x6d799b5 (0xc3b6 bytes) 2024-09-17T21:48:13.471 INFO:teuthology.orchestra.run.smithi165.stdout:4744 read 0x82eba00 thru 0x82fb1c4 (0xf7c5 bytes) 2024-09-17T21:48:13.474 INFO:teuthology.orchestra.run.smithi165.stdout:4746 read 0x4f96600 thru 0x4f9f22a (0x8c2b bytes) 2024-09-17T21:48:13.474 INFO:teuthology.orchestra.run.smithi165.stdout:4747 read 0x6999a00 thru 0x69a42e7 (0xa8e8 bytes) 2024-09-17T21:48:13.477 INFO:teuthology.orchestra.run.smithi165.stdout:4750 trunc from 0x9ff8000 to 0x5400200 2024-09-17T21:48:13.493 INFO:teuthology.orchestra.run.smithi165.stdout:4751 punch from 0x4188200 to 0x4190e00, (0x8c00 bytes) 2024-09-17T21:48:13.498 INFO:teuthology.orchestra.run.smithi165.stdout:4753 trunc from 0x5400200 to 0x5eae200 2024-09-17T21:48:13.508 INFO:teuthology.orchestra.run.smithi165.stdout:4754 trunc from 0x5eae200 to 0x8e05a00 2024-09-17T21:48:13.534 INFO:teuthology.orchestra.run.smithi165.stdout:4759 write 0xa1ad200 thru 0xa1bc04e (0xee4f bytes) 2024-09-17T21:48:13.536 INFO:teuthology.orchestra.run.smithi165.stdout:4760 write 0x916d000 thru 0x917758d (0xa58e bytes) 2024-09-17T21:48:13.539 INFO:teuthology.orchestra.run.smithi165.stdout:4761 trunc from 0xa1bc200 to 0xc808000 2024-09-17T21:48:13.544 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:13.541+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:13.551 INFO:teuthology.orchestra.run.smithi165.stdout:4764 punch from 0x1527200 to 0x152a000, (0x2e00 bytes) 2024-09-17T21:48:13.553 INFO:teuthology.orchestra.run.smithi165.stdout:4765 punch from 0x7e4c400 to 0x7e53e00, (0x7a00 bytes) 2024-09-17T21:48:13.553 INFO:teuthology.orchestra.run.smithi165.stdout:4767 punch from 0x6503600 to 0x6509e00, (0x6800 bytes) 2024-09-17T21:48:13.556 INFO:teuthology.orchestra.run.smithi165.stdout:4768 write 0x54c1c00 thru 0x54cb380 (0x9781 bytes) 2024-09-17T21:48:13.556 INFO:teuthology.orchestra.run.smithi165.stdout:4770 punch from 0x376d800 to 0x376fc00, (0x2400 bytes) 2024-09-17T21:48:13.568 INFO:teuthology.orchestra.run.smithi165.stdout:4772 write 0xd9ff200 thru 0xda0619e (0x6f9f bytes) 2024-09-17T21:48:13.568 INFO:teuthology.orchestra.run.smithi165.stdout:4774 read 0x7e49200 thru 0x7e5603d (0xce3e bytes) 2024-09-17T21:48:13.569 INFO:teuthology.orchestra.run.smithi165.stdout:4778 write 0x6c43c00 thru 0x6c45cd7 (0x20d8 bytes) 2024-09-17T21:48:13.569 INFO:teuthology.orchestra.run.smithi165.stdout:4779 read 0xa95c800 thru 0xa95c8fa (0xfb bytes) 2024-09-17T21:48:13.570 INFO:teuthology.orchestra.run.smithi165.stdout:4783 read 0x9ba3000 thru 0x9ba89fe (0x59ff bytes) 2024-09-17T21:48:13.570 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:13.570 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:13.571 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:13.571 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:13.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:13.585 INFO:teuthology.orchestra.run.smithi165.stdout:4784 trunc from 0xda06200 to 0x257400 2024-09-17T21:48:13.625 INFO:teuthology.orchestra.run.smithi165.stdout:4785 trunc from 0x257400 to 0xdbab000 2024-09-17T21:48:13.634 INFO:teuthology.orchestra.run.smithi165.stdout:4787 read 0x728d200 thru 0x729c553 (0xf354 bytes) 2024-09-17T21:48:13.636 INFO:teuthology.orchestra.run.smithi165.stdout:4788 punch from 0x494dc00 to 0x4953a00, (0x5e00 bytes) 2024-09-17T21:48:13.637 INFO:teuthology.orchestra.run.smithi165.stdout:4789 read 0x6185200 thru 0x618ed8d (0x9b8e bytes) 2024-09-17T21:48:13.638 INFO:teuthology.orchestra.run.smithi165.stdout:4790 punch from 0x18dde00 to 0x18e1800, (0x3a00 bytes) 2024-09-17T21:48:13.638 INFO:teuthology.orchestra.run.smithi165.stdout:4792 punch from 0xcb4d800 to 0xcb4e200, (0xa00 bytes) 2024-09-17T21:48:13.650 INFO:teuthology.orchestra.run.smithi165.stdout:4793 trunc from 0xdbab000 to 0x3421000 2024-09-17T21:48:13.663 INFO:teuthology.orchestra.run.smithi165.stdout:4794 punch from 0x9b8200 to 0x9c3400, (0xb200 bytes) 2024-09-17T21:48:13.664 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:13.661+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:13.680 INFO:teuthology.orchestra.run.smithi165.stdout:4795 trunc from 0x3421000 to 0xec23800 2024-09-17T21:48:13.689 INFO:teuthology.orchestra.run.smithi165.stdout:4796 write 0xae3c00 thru 0xaeb092 (0x7493 bytes) 2024-09-17T21:48:13.695 INFO:teuthology.orchestra.run.smithi165.stdout:4797 write 0x56a1e00 thru 0x56ab64e (0x984f bytes) 2024-09-17T21:48:13.695 INFO:teuthology.orchestra.run.smithi165.stdout:4798 read 0x9420000 thru 0x942900c (0x900d bytes) 2024-09-17T21:48:13.695 INFO:teuthology.orchestra.run.smithi165.stdout:4799 read 0xf5b000 thru 0xf66a9e (0xba9f bytes) 2024-09-17T21:48:13.696 INFO:teuthology.orchestra.run.smithi165.stdout:4804 write 0x4cffa00 thru 0x4d00d8d (0x138e bytes) 2024-09-17T21:48:13.696 INFO:teuthology.orchestra.run.smithi165.stdout:4805 punch from 0x6cf600 to 0x6d8400, (0x8e00 bytes) 2024-09-17T21:48:13.696 INFO:teuthology.orchestra.run.smithi165.stdout:4806 read 0x324c800 thru 0x3258f99 (0xc79a bytes) 2024-09-17T21:48:13.713 INFO:teuthology.orchestra.run.smithi165.stdout:4807 trunc from 0xec23800 to 0x144200 2024-09-17T21:48:13.735 INFO:teuthology.orchestra.run.smithi165.stdout:4808 read 0xdb800 thru 0xe136a (0x5b6b bytes) 2024-09-17T21:48:13.737 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:13.737 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:13.737 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:13.741 INFO:teuthology.orchestra.run.smithi165.stdout:4809 punch from 0x9b600 to 0xa4a00, (0x9400 bytes) 2024-09-17T21:48:13.741 INFO:teuthology.orchestra.run.smithi165.stdout:4811 read 0x69600 thru 0x6c605 (0x3006 bytes) 2024-09-17T21:48:13.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:13.747 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:13.743+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:13.757 INFO:teuthology.orchestra.run.smithi165.stdout:4812 trunc from 0x144200 to 0xc7c7400 2024-09-17T21:48:13.763 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:13.771 INFO:teuthology.orchestra.run.smithi165.stdout:4814 trunc from 0xc7c7400 to 0x8345200 2024-09-17T21:48:13.796 INFO:teuthology.orchestra.run.smithi165.stdout:4816 write 0xdc06a00 thru 0xdc16480 (0xfa81 bytes) 2024-09-17T21:48:13.801 INFO:teuthology.orchestra.run.smithi165.stdout:4818 read 0x2dfe200 thru 0x2e0e07e (0xfe7f bytes) 2024-09-17T21:48:13.801 INFO:teuthology.orchestra.run.smithi165.stdout:4819 punch from 0xa7dac00 to 0xa7e2000, (0x7400 bytes) 2024-09-17T21:48:13.803 INFO:teuthology.orchestra.run.smithi165.stdout:4821 read 0x2ab9e00 thru 0x2ac7af3 (0xdcf4 bytes) 2024-09-17T21:48:13.804 INFO:teuthology.orchestra.run.smithi165.stdout:4822 punch from 0xc459000 to 0xc466e00, (0xde00 bytes) 2024-09-17T21:48:13.804 INFO:teuthology.orchestra.run.smithi165.stdout:4825 punch from 0x35fe400 to 0x35ff800, (0x1400 bytes) 2024-09-17T21:48:13.804 INFO:teuthology.orchestra.run.smithi165.stdout:4827 write 0x5181200 thru 0x5189f0a (0x8d0b bytes) 2024-09-17T21:48:13.804 INFO:teuthology.orchestra.run.smithi165.stdout:4828 punch from 0xaf1ee00 to 0xaf29400, (0xa600 bytes) 2024-09-17T21:48:13.804 INFO:teuthology.orchestra.run.smithi165.stdout:4830 clone 40 order 22 su 131072 sc 2 2024-09-17T21:48:13.847 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:13.844+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.214066714, 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.181791739, 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:13.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.181552971, 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:13.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:13.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.014046841000003, 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:13.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:13.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:13.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.00714507, 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:13.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.007028978999998, 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:13.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:13.934 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.810025458999998, 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:13.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:13.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:13.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809986664, 2024-09-17T21:48:13.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:13.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:13.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:13.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809972380000001, 2024-09-17T21:48:13.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:13.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:13.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809727414000001, 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:13.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809644966, 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:13.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:13.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:13.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809613156000001, 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:13.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.809589437, 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:13.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:13.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:13.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.42295004, 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:13.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 24.413021059999998, 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:13.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2206863459999999, 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:13.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:13.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:13.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:13.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2204289250000002, 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:13.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2203960790000004, 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:13.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:13.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:13.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:13.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2203791339999999, 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.998 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:13.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:14.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:14.003 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.2203376370000001, 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:14.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:14.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:14.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:14.009 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:14.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:14.011 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:14.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:14.012 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:14.012 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:14.013 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:14.013 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:14.014 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:14.014 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:14.014 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:14.015 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:14.015 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:14.007+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:14.111 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:14.108+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:14.159 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.159 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:14.251 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:14.248+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:14.312 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:14.309+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:14.314 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.314 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:14.413 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:14.411+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:14.470 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.471 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:14.524 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:14.521+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:14.587 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.588 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:14.637 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:14.634+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:14.716 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:14.712+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:14.719 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:48:14.746 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.746 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.756 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:14.779 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone39 from 0xdc16600 (overlap 0x144200) to 0x40c00 2024-09-17T21:48:14.817 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:14.814+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:14.908 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:14.908 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:14.915 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:14.917 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:14.914+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:15.018 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:15.014+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:15.026 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.026 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:15.119 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:15.116+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:15.148 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.149 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:15.158 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-17T21:48:15.220 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:15.216+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:15.270 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.270 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:15.320 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:15.317+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:15.397 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.397 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:15.450 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:15.448+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:15.522 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:15.519+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:15.523 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.523 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.532 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:15.623 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:15.621+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:15.680 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.680 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:48:15.718 INFO:teuthology.orchestra.run.smithi165.stdout:4831 write 0x4298400 thru 0x42a3c49 (0xb84a bytes) 2024-09-17T21:48:15.721 INFO:teuthology.orchestra.run.smithi165.stdout:4833 write 0xc3dac00 thru 0xc3e899b (0xdd9c bytes) 2024-09-17T21:48:15.723 INFO:teuthology.orchestra.run.smithi165.stdout:4834 punch from 0x3e82000 to 0x3e85a00, (0x3a00 bytes) 2024-09-17T21:48:15.723 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:15.721+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:15.750 INFO:teuthology.orchestra.run.smithi165.stdout:4836 trunc from 0xdc16600 to 0x27a4000 2024-09-17T21:48:15.768 INFO:teuthology.orchestra.run.smithi165.stdout:4839 read 0x1f6a00 thru 0x1fc60e (0x5c0f bytes) 2024-09-17T21:48:15.769 INFO:teuthology.orchestra.run.smithi165.stdout:4841 read 0x7a000 thru 0x899c1 (0xf9c2 bytes) 2024-09-17T21:48:15.773 INFO:teuthology.orchestra.run.smithi165.stdout:4842 read 0x226d000 thru 0x227a5aa (0xd5ab bytes) 2024-09-17T21:48:15.795 INFO:teuthology.orchestra.run.smithi165.stdout:4843 write 0xcc9b200 thru 0xcca85d1 (0xd3d2 bytes) 2024-09-17T21:48:15.796 INFO:teuthology.orchestra.run.smithi165.stdout:4845 read 0x9cf0400 thru 0x9cf6ec2 (0x6ac3 bytes) 2024-09-17T21:48:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.803 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.809 INFO:teuthology.orchestra.run.smithi165.stdout:4847 trunc from 0xcca8600 to 0x17e2600 2024-09-17T21:48:15.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:15.825 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:15.822+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:15.851 INFO:teuthology.orchestra.run.smithi165.stdout:4848 write 0xcd23600 thru 0xcd2c722 (0x9123 bytes) 2024-09-17T21:48:15.859 INFO:teuthology.orchestra.run.smithi165.stdout:4852 trunc from 0xcd2c800 to 0x7a9ec00 2024-09-17T21:48:15.875 INFO:teuthology.orchestra.run.smithi165.stdout:4855 punch from 0x3be3a00 to 0x3be7800, (0x3e00 bytes) 2024-09-17T21:48:15.877 INFO:teuthology.orchestra.run.smithi165.stdout:4856 read 0x5d3200 thru 0x5de065 (0xae66 bytes) 2024-09-17T21:48:15.878 INFO:teuthology.orchestra.run.smithi165.stdout:4857 trunc from 0x7a9ec00 to 0x734b600 2024-09-17T21:48:15.889 INFO:teuthology.orchestra.run.smithi165.stdout:4858 punch from 0x3b26200 to 0x3b26c00, (0xa00 bytes) 2024-09-17T21:48:15.892 INFO:teuthology.orchestra.run.smithi165.stdout:4864 punch from 0x1a09400 to 0x1a16e00, (0xda00 bytes) 2024-09-17T21:48:15.892 INFO:teuthology.orchestra.run.smithi165.stdout:4865 read 0x43b400 thru 0x444c45 (0x9846 bytes) 2024-09-17T21:48:15.898 INFO:teuthology.orchestra.run.smithi165.stdout:4866 trunc from 0x734b600 to 0xb940200 2024-09-17T21:48:15.913 INFO:teuthology.orchestra.run.smithi165.stdout:4867 trunc from 0xb940200 to 0x6486e00 2024-09-17T21:48:15.922 INFO:teuthology.orchestra.run.smithi165.stdout:4869 punch from 0x4bcb600 to 0x4bd4400, (0x8e00 bytes) 2024-09-17T21:48:15.923 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:15.924 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:15.924 INFO:teuthology.orchestra.run.smithi165.stdout:4874 punch from 0x4677e00 to 0x467ae00, (0x3000 bytes) 2024-09-17T21:48:15.925 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:15.922+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:15.926 INFO:teuthology.orchestra.run.smithi165.stdout:4875 trunc from 0x6486e00 to 0x4ff0c00 2024-09-17T21:48:15.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:15.942 INFO:teuthology.orchestra.run.smithi165.stdout:4876 punch from 0x2dc7400 to 0x2dd2e00, (0xba00 bytes) 2024-09-17T21:48:15.944 INFO:teuthology.orchestra.run.smithi165.stdout:4877 trunc from 0x4ff0c00 to 0x3a0c800 2024-09-17T21:48:15.965 INFO:teuthology.orchestra.run.smithi165.stdout:4879 punch from 0x3007c00 to 0x3010e00, (0x9200 bytes) 2024-09-17T21:48:15.965 INFO:teuthology.orchestra.run.smithi165.stdout:4881 read 0x1879000 thru 0x187d3ed (0x43ee bytes) 2024-09-17T21:48:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:4882 write 0xca14000 thru 0xca1678b (0x278c bytes) 2024-09-17T21:48:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:4884 write 0x9e25200 thru 0x9e33b40 (0xe941 bytes) 2024-09-17T21:48:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:4886 punch from 0x5bbfe00 to 0x5bc8000, (0x8200 bytes) 2024-09-17T21:48:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:4887 write 0x33f1800 thru 0x33ffdec (0xe5ed bytes) 2024-09-17T21:48:15.991 INFO:teuthology.orchestra.run.smithi165.stdout:4890 punch from 0x7536e00 to 0x7538000, (0x1200 bytes) 2024-09-17T21:48:16.001 INFO:teuthology.orchestra.run.smithi165.stdout:4891 trunc from 0xca16800 to 0x42fa800 2024-09-17T21:48:16.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:16.016 INFO:teuthology.orchestra.run.smithi165.stdout:4892 punch from 0x2ad9800 to 0x2ae6c00, (0xd400 bytes) 2024-09-17T21:48:16.025 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:16.023+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:16.032 INFO:teuthology.orchestra.run.smithi165.stdout:4893 write 0x95a3000 thru 0x95a9f7f (0x6f80 bytes) 2024-09-17T21:48:16.033 INFO:teuthology.orchestra.run.smithi165.stdout:4895 read 0x5201c00 thru 0x520aafa (0x8efb bytes) 2024-09-17T21:48:16.035 INFO:teuthology.orchestra.run.smithi165.stdout:4896 trunc from 0x95aa000 to 0xb25ce00 2024-09-17T21:48:16.048 INFO:teuthology.orchestra.run.smithi165.stdout:4898 read 0x920ce00 thru 0x9219fc8 (0xd1c9 bytes) 2024-09-17T21:48:16.048 INFO:teuthology.orchestra.run.smithi165.stdout:4900 write 0x7243e00 thru 0x7246566 (0x2767 bytes) 2024-09-17T21:48:16.048 INFO:teuthology.orchestra.run.smithi165.stdout:4902 read 0x622ae00 thru 0x62396cd (0xe8ce bytes) 2024-09-17T21:48:16.049 INFO:teuthology.orchestra.run.smithi165.stdout:4903 clone 41 order 23 su 131072 sc 5 2024-09-17T21:48:16.087 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:16.088 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:16.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:16.126 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:16.124+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:16.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:16.144 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:16.161 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:16.162 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:16.178 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:16.195 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:16.196 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:16.208 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:16.208 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:16.212 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:16.212 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:16.217 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:16.217 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:16.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:16.261 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:16.258+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:16.325 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:16.325 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:16.325 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:16.325 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:16.326 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:16.328 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:16.326+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:16.336 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:16.429 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:16.426+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:16.483 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:16.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.396951951999998, 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.484 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:16.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:16.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395809664000002, 2024-09-17T21:48:16.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:16.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:16.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395793618999999, 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:16.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:16.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395777074000002, 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:16.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395762308999998, 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:16.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395735003999999, 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:16.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395698115999998, 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:16.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395659418000001, 2024-09-17T21:48:16.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:16.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395621245000001, 2024-09-17T21:48:16.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:16.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395583815999998, 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:16.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:16.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395526792999998, 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:16.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:16.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:16.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395496123000001, 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:16.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:16.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.39535175, 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:16.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395332905, 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:16.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395306308999999, 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:16.561 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 18.395271470000001, 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.820768503, 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:16.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.820735188, 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:16.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:16.577 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.813645124000001, 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:16.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:16.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 12.813611443999999, 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:16.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:16.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:16.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:16.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.588 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:16.588 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:16.588 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:16.589 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:16.590 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:16.586+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:16.690 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:16.687+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:16.741 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:16.741 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:16.751 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:16.795 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:16.792+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:16.805 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone40 from 0xb25ce00 (overlap 0x17e2600) to 0x4c6e00 2024-09-17T21:48:16.863 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:16.863 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:16.864 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:16.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:16.913 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:16.911+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:16.992 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:16.989+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 102.147001718, 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:17.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:17.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:17.029 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110555408, 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110534106999999, 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:17.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:17.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110513346000001, 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:17.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:17.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110492239999999, 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:17.043 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:17.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110471750999999, 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:17.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:17.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110447654000001, 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110425160999998, 2024-09-17T21:48:17.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:17.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:17.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110397553999999, 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:17.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.110372714, 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.958418939000001, 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:17.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.958395177, 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:17.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:17.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.958352787999999, 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:17.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902604135000001, 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:17.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902581040000001, 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.90255466, 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:17.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:17.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:17.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:17.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902538024000002, 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:17.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902508059999999, 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:17.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:17.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:17.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902495134999999, 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 25.902484913999999, 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.112 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:17.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:17.113 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:17.114 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:17.115 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:17.111+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:17.152 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-17T21:48:17.215 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:17.212+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:17.223 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:17.223 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:17.224 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:17.225 INFO:teuthology.orchestra.run.smithi165.stdout:4904 trunc from 0xb25ce00 to 0x922000 2024-09-17T21:48:17.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:17.266 INFO:teuthology.orchestra.run.smithi165.stdout:4908 trunc from 0x922000 to 0x4274000 2024-09-17T21:48:17.274 INFO:teuthology.orchestra.run.smithi165.stdout:4909 read 0x1c61000 thru 0x1c6ce6e (0xbe6f bytes) 2024-09-17T21:48:17.276 INFO:teuthology.orchestra.run.smithi165.stdout:4914 read 0x12ab600 thru 0x12ad555 (0x1f56 bytes) 2024-09-17T21:48:17.277 INFO:teuthology.orchestra.run.smithi165.stdout:4915 read 0x1e2d000 thru 0x1e3ae59 (0xde5a bytes) 2024-09-17T21:48:17.288 INFO:teuthology.orchestra.run.smithi165.stdout:4916 trunc from 0x4274000 to 0xd591600 2024-09-17T21:48:17.303 INFO:teuthology.orchestra.run.smithi165.stdout:4918 write 0x10e4000 thru 0x10e7203 (0x3204 bytes) 2024-09-17T21:48:17.303 INFO:teuthology.orchestra.run.smithi165.stdout:4920 write 0x787e400 thru 0x788b5b7 (0xd1b8 bytes) 2024-09-17T21:48:17.307 INFO:teuthology.orchestra.run.smithi165.stdout:4921 trunc from 0xd591600 to 0xa134800 2024-09-17T21:48:17.317 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:17.314+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:17.326 INFO:teuthology.orchestra.run.smithi165.stdout:4922 trunc from 0xa134800 to 0x5dd3000 2024-09-17T21:48:17.356 INFO:teuthology.orchestra.run.smithi165.stdout:4924 write 0xa5a0a00 thru 0xa5abce0 (0xb2e1 bytes) 2024-09-17T21:48:17.357 INFO:teuthology.orchestra.run.smithi165.stdout:4926 write 0x8b2a200 thru 0x8b2becc (0x1ccd bytes) 2024-09-17T21:48:17.357 INFO:teuthology.orchestra.run.smithi165.stdout:4927 read 0x810d000 thru 0x8113dbf (0x6dc0 bytes) 2024-09-17T21:48:17.364 INFO:teuthology.orchestra.run.smithi165.stdout:4928 trunc from 0xa5abe00 to 0x446d000 2024-09-17T21:48:17.388 INFO:teuthology.orchestra.run.smithi165.stdout:4930 read 0x134d600 thru 0x1352826 (0x5227 bytes) 2024-09-17T21:48:17.389 INFO:teuthology.orchestra.run.smithi165.stdout:4931 write 0x169a00 thru 0x16e573 (0x4b74 bytes) 2024-09-17T21:48:17.390 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:17.390 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:17.390 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:17.390 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:17.390 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:17.393 INFO:teuthology.orchestra.run.smithi165.stdout:4938 trunc from 0x446d000 to 0xfca800 2024-09-17T21:48:17.400 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:17.413 INFO:teuthology.orchestra.run.smithi165.stdout:4940 read 0x1a5e00 thru 0x1ab7ca (0x59cb bytes) 2024-09-17T21:48:17.431 INFO:teuthology.orchestra.run.smithi165.stdout:4941 write 0x154a800 thru 0x15500cf (0x58d0 bytes) 2024-09-17T21:48:17.436 INFO:teuthology.orchestra.run.smithi165.stdout:4942 trunc from 0x1550200 to 0x4f11e00 2024-09-17T21:48:17.444 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:17.442+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:17.448 INFO:teuthology.orchestra.run.smithi165.stdout:4944 read 0x2cca600 thru 0x2ccea3d (0x443e bytes) 2024-09-17T21:48:17.451 INFO:teuthology.orchestra.run.smithi165.stdout:4945 read 0x3cc2200 thru 0x3cc4ceb (0x2aec bytes) 2024-09-17T21:48:17.451 INFO:teuthology.orchestra.run.smithi165.stdout:4947 read 0x41d0800 thru 0x41db7e1 (0xafe2 bytes) 2024-09-17T21:48:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:4948 write 0xc54de00 thru 0xc55a675 (0xc876 bytes) 2024-09-17T21:48:17.474 INFO:teuthology.orchestra.run.smithi165.stdout:4949 read 0x299f600 thru 0x29a012b (0xb2c bytes) 2024-09-17T21:48:17.475 INFO:teuthology.orchestra.run.smithi165.stdout:4951 punch from 0x68f8400 to 0x6901000, (0x8c00 bytes) 2024-09-17T21:48:17.475 INFO:teuthology.orchestra.run.smithi165.stdout:4952 read 0x150000 thru 0x1558c1 (0x58c2 bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4953 write 0x35d2a00 thru 0x35e1c9d (0xf29e bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4954 write 0xa4d5e00 thru 0xa4e4ccb (0xeecc bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4955 punch from 0x9f4ae00 to 0x9f50000, (0x5200 bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4957 punch from 0x3102600 to 0x3112200, (0xfc00 bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4959 read 0x92d6800 thru 0x92e1353 (0xab54 bytes) 2024-09-17T21:48:17.476 INFO:teuthology.orchestra.run.smithi165.stdout:4961 punch from 0x25c8c00 to 0x25d1a00, (0x8e00 bytes) 2024-09-17T21:48:17.489 INFO:teuthology.orchestra.run.smithi165.stdout:4962 trunc from 0xc55a800 to 0xb69c00 2024-09-17T21:48:17.519 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:17.515+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:17.520 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:17.520 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.777446055, 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:17.521 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:17.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 111.776401595, 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:17.526 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:17.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.575216263999998, 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:17.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:17.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:17.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.359298595999999, 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:17.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:17.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156861853999999, 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:17.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:17.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:17.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156804674, 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.551 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:17.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:17.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156234092999998, 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:17.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:17.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156206159, 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:17.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156071807, 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:17.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:17.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:17.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.156050272000002, 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:17.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.155889306999999, 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.155833542, 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.154486935000001, 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.154435378999999, 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.153747117999998, 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.153730067000001, 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.152791827000001, 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.152761486999999, 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.858364039, 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.957246338999999, 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:17.635 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:17.637 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4966 write 0xde34200 thru 0xde43978 (0xf779 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4967 punch from 0x9670600 to 0x9670a00, (0x400 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4969 write 0x9f9b400 thru 0x9fa80ab (0xccac bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4970 read 0x4f19200 thru 0x4f20d42 (0x7b43 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4971 write 0xbacaa00 thru 0xbada254 (0xf855 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4973 trunc from 0xde43a00 to 0x88ce600 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4974 read 0x5f5b600 thru 0x5f5d987 (0x2388 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4975 read 0x6738e00 thru 0x67433d3 (0xa5d4 bytes) 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4976 trunc from 0x88ce600 to 0xd8adc00 2024-09-17T21:48:17.639 INFO:teuthology.orchestra.run.smithi165.stdout:4979 punch from 0x10f5600 to 0x10fa200, (0x4c00 bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4980 read 0x1364400 thru 0x1371c5d (0xd85e bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4981 punch from 0xd88800 to 0xd8e000, (0x5800 bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4982 write 0x9430600 thru 0x9430b56 (0x557 bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4984 write 0x1e03c00 thru 0x1e080fb (0x44fc bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4986 trunc from 0xd8adc00 to 0x8e54400 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4987 punch from 0x2a37e00 to 0x2a3a600, (0x2800 bytes) 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4988 trunc from 0x8e54400 to 0x351fc00 2024-09-17T21:48:17.640 INFO:teuthology.orchestra.run.smithi165.stdout:4989 read 0x337400 thru 0x34058e (0x918f bytes) 2024-09-17T21:48:17.640 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:17.633+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:17.643 INFO:teuthology.orchestra.run.smithi165.stdout:4990 read 0x1df2600 thru 0x1df531f (0x2d20 bytes) 2024-09-17T21:48:17.643 INFO:teuthology.orchestra.run.smithi165.stdout:4992 read 0x2a87400 thru 0x2a94011 (0xcc12 bytes) 2024-09-17T21:48:17.656 INFO:teuthology.orchestra.run.smithi165.stdout:4994 write 0x6a6e800 thru 0x6a708e6 (0x20e7 bytes) 2024-09-17T21:48:17.658 INFO:teuthology.orchestra.run.smithi165.stdout:4997 punch from 0x5874600 to 0x587da00, (0x9400 bytes) 2024-09-17T21:48:17.662 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 4, 0] out_osds: [2] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:48:17.662 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:17.662 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-17T21:48:17.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:48:17.663 INFO:teuthology.orchestra.run.smithi165.stdout:4999 read 0x4a3dc00 thru 0x4a4adc9 (0xd1ca bytes) 2024-09-17T21:48:17.663 INFO:teuthology.orchestra.run.smithi165.stdout:5001 read 0x2ce7600 thru 0x2cece04 (0x5805 bytes) 2024-09-17T21:48:17.663 INFO:teuthology.orchestra.run.smithi165.stdout:5003 write 0x2e1d200 thru 0x2e2a9ef (0xd7f0 bytes) 2024-09-17T21:48:17.680 INFO:teuthology.orchestra.run.smithi165.stdout:5004 write 0xaed2000 thru 0xaedafb8 (0x8fb9 bytes) 2024-09-17T21:48:17.681 INFO:teuthology.orchestra.run.smithi165.stdout:5005 punch from 0x715f800 to 0x716a400, (0xac00 bytes) 2024-09-17T21:48:17.684 INFO:teuthology.orchestra.run.smithi165.stdout:5010 write 0x800c200 thru 0x8012f25 (0x6d26 bytes) 2024-09-17T21:48:17.684 INFO:teuthology.orchestra.run.smithi165.stdout:5013 punch from 0x6924c00 to 0x6931600, (0xca00 bytes) 2024-09-17T21:48:17.684 INFO:teuthology.orchestra.run.smithi165.stdout:5015 write 0x1e47200 thru 0x1e4a960 (0x3761 bytes) 2024-09-17T21:48:17.685 INFO:teuthology.orchestra.run.smithi165.stdout:5019 read 0x55cd000 thru 0x55d74f2 (0xa4f3 bytes) 2024-09-17T21:48:17.700 INFO:teuthology.orchestra.run.smithi165.stdout:5020 write 0xd0a0e00 thru 0xd0a61e1 (0x53e2 bytes) 2024-09-17T21:48:17.700 INFO:teuthology.orchestra.run.smithi165.stdout:5021 read 0xab34c00 thru 0xab40d0a (0xc10b bytes) 2024-09-17T21:48:17.701 INFO:teuthology.orchestra.run.smithi165.stdout:5022 write 0x844fc00 thru 0x845b6fa (0xbafb bytes) 2024-09-17T21:48:17.701 INFO:teuthology.orchestra.run.smithi165.stdout:5023 punch from 0x66dc400 to 0x66e2c00, (0x6800 bytes) 2024-09-17T21:48:17.703 INFO:teuthology.orchestra.run.smithi165.stdout:5026 read 0x5aa8a00 thru 0x5ab88eb (0xfeec bytes) 2024-09-17T21:48:17.703 INFO:teuthology.orchestra.run.smithi165.stdout:5027 write 0x8627800 thru 0x86319d4 (0xa1d5 bytes) 2024-09-17T21:48:17.706 INFO:teuthology.orchestra.run.smithi165.stdout:5028 trunc from 0xd0a6200 to 0xa8be400 2024-09-17T21:48:17.722 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:48:17.723 INFO:teuthology.orchestra.run.smithi165.stdout:5029 read 0x5dd6000 thru 0x5dda488 (0x4489 bytes) 2024-09-17T21:48:17.724 INFO:teuthology.orchestra.run.smithi165.stdout:5030 read 0x36f000 thru 0x379ef0 (0xaef1 bytes) 2024-09-17T21:48:17.728 INFO:teuthology.orchestra.run.smithi165.stdout:5031 read 0x890a400 thru 0x890cb25 (0x2726 bytes) 2024-09-17T21:48:17.728 INFO:teuthology.orchestra.run.smithi165.stdout:5032 read 0x4427600 thru 0x44359ab (0xe3ac bytes) 2024-09-17T21:48:17.728 INFO:teuthology.orchestra.run.smithi165.stdout:5039 read 0x655da00 thru 0x65699bb (0xbfbc bytes) 2024-09-17T21:48:17.733 INFO:teuthology.orchestra.run.smithi165.stdout:5040 trunc from 0xa8be400 to 0x3228800 2024-09-17T21:48:17.751 INFO:teuthology.orchestra.run.smithi165.stdout:5041 read 0x564400 thru 0x571503 (0xd104 bytes) 2024-09-17T21:48:17.768 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:17.763+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:17.769 INFO:teuthology.orchestra.run.smithi165.stdout:5042 write 0x7245c00 thru 0x7250bce (0xafcf bytes) 2024-09-17T21:48:17.776 INFO:teuthology.orchestra.run.smithi165.stdout:5043 trunc from 0x7250c00 to 0xd105400 2024-09-17T21:48:17.789 INFO:teuthology.orchestra.run.smithi165.stdout:5044 write 0x7ac0800 thru 0x7ac428a (0x3a8b bytes) 2024-09-17T21:48:17.790 INFO:teuthology.orchestra.run.smithi165.stdout:5045 read 0x794e00 thru 0x7a343d (0xe63e bytes) 2024-09-17T21:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout:5047 write 0xc016400 thru 0xc022f4d (0xcb4e bytes) 2024-09-17T21:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout:5048 write 0x8828600 thru 0x882ec18 (0x6619 bytes) 2024-09-17T21:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout:5049 read 0xc088800 thru 0xc095942 (0xd143 bytes) 2024-09-17T21:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout:5051 write 0x4670800 thru 0x46771f1 (0x69f2 bytes) 2024-09-17T21:48:17.793 INFO:teuthology.orchestra.run.smithi165.stdout:5054 read 0x8840200 thru 0x88495df (0x93e0 bytes) 2024-09-17T21:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout:5055 write 0xd01ac00 thru 0xd024ea6 (0xa2a7 bytes) 2024-09-17T21:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout:5057 write 0xa1eae00 thru 0xa1ee5c6 (0x37c7 bytes) 2024-09-17T21:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout:5058 punch from 0x66000 to 0x73400, (0xd400 bytes) 2024-09-17T21:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout:5059 read 0xcae3400 thru 0xcae35b3 (0x1b4 bytes) 2024-09-17T21:48:17.794 INFO:teuthology.orchestra.run.smithi165.stdout:5060 write 0x20f1e00 thru 0x20f9b39 (0x7d3a bytes) 2024-09-17T21:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout:5061 read 0xb645e00 thru 0xb64d23a (0x743b bytes) 2024-09-17T21:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout:5063 write 0x9417800 thru 0x9423abe (0xc2bf bytes) 2024-09-17T21:48:17.796 INFO:teuthology.orchestra.run.smithi165.stdout:5066 punch from 0xcaada00 to 0xcaae800, (0xe00 bytes) 2024-09-17T21:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout:5067 read 0x19f8800 thru 0x19f9d89 (0x158a bytes) 2024-09-17T21:48:17.797 INFO:teuthology.orchestra.run.smithi165.stdout:5071 write 0x5831800 thru 0x583a9a3 (0x91a4 bytes) 2024-09-17T21:48:17.803 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:17.803 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:17.803 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:17.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:17.819 INFO:teuthology.orchestra.run.smithi165.stdout:5073 write 0xd654800 thru 0xd65d713 (0x8f14 bytes) 2024-09-17T21:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout:5074 read 0xb3d3800 thru 0xb3d8bfa (0x53fb bytes) 2024-09-17T21:48:17.820 INFO:teuthology.orchestra.run.smithi165.stdout:5075 write 0x251bc00 thru 0x251ecd0 (0x30d1 bytes) 2024-09-17T21:48:17.823 INFO:teuthology.orchestra.run.smithi165.stdout:5076 write 0xa957a00 thru 0xa95db11 (0x6112 bytes) 2024-09-17T21:48:17.824 INFO:teuthology.orchestra.run.smithi165.stdout:5078 punch from 0x1019c00 to 0x101a800, (0xc00 bytes) 2024-09-17T21:48:17.831 INFO:teuthology.orchestra.run.smithi165.stdout:5080 trunc from 0xd65d800 to 0x5deca00 2024-09-17T21:48:17.838 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:17.835+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:17.849 INFO:teuthology.orchestra.run.smithi165.stdout:5082 read 0x1c44000 thru 0x1c48a86 (0x4a87 bytes) 2024-09-17T21:48:17.872 INFO:teuthology.orchestra.run.smithi165.stdout:5085 write 0xe8fe200 thru 0xe909f80 (0xbd81 bytes) 2024-09-17T21:48:17.875 INFO:teuthology.orchestra.run.smithi165.stdout:5086 write 0x36d2600 thru 0x36df6ed (0xd0ee bytes) 2024-09-17T21:48:17.876 INFO:teuthology.orchestra.run.smithi165.stdout:5087 write 0x5bd6800 thru 0x5bdce77 (0x6678 bytes) 2024-09-17T21:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout:5089 read 0xda21a00 thru 0xda2555e (0x3b5f bytes) 2024-09-17T21:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout:5090 read 0x13e3400 thru 0x13ed427 (0xa028 bytes) 2024-09-17T21:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout:5092 write 0x67f7000 thru 0x67fbeb6 (0x4eb7 bytes) 2024-09-17T21:48:17.877 INFO:teuthology.orchestra.run.smithi165.stdout:5096 read 0x50f7200 thru 0x5105496 (0xe297 bytes) 2024-09-17T21:48:17.892 INFO:teuthology.orchestra.run.smithi165.stdout:5097 trunc from 0xe90a000 to 0xefe200 2024-09-17T21:48:17.923 INFO:teuthology.orchestra.run.smithi165.stdout:5100 punch from 0x31e200 to 0x31fe00, (0x1c00 bytes) 2024-09-17T21:48:17.924 INFO:teuthology.orchestra.run.smithi165.stdout:5101 read 0x3e8600 thru 0x3e8af4 (0x4f5 bytes) 2024-09-17T21:48:17.924 INFO:teuthology.orchestra.run.smithi165.stdout:5102 read 0xe73c00 thru 0xe7b73b (0x7b3c bytes) 2024-09-17T21:48:17.926 INFO:teuthology.orchestra.run.smithi165.stdout:5112 trunc from 0xefe200 to 0x775800 2024-09-17T21:48:17.929 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:17.929 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:17.930 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:17.930 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:17.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:17.939 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:17.936+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:17.943 INFO:teuthology.orchestra.run.smithi165.stdout:5113 read 0x5b8800 thru 0x5c8701 (0xff02 bytes) 2024-09-17T21:48:17.948 INFO:teuthology.orchestra.run.smithi165.stdout:5114 trunc from 0x775800 to 0x4dc5800 2024-09-17T21:48:17.964 INFO:teuthology.orchestra.run.smithi165.stdout:5115 read 0x2557400 thru 0x255e91a (0x751b bytes) 2024-09-17T21:48:17.989 INFO:teuthology.orchestra.run.smithi165.stdout:5116 write 0xc520200 thru 0xc52c2e0 (0xc0e1 bytes) 2024-09-17T21:48:17.989 INFO:teuthology.orchestra.run.smithi165.stdout:5117 write 0x5ca4400 thru 0x5ca865d (0x425e bytes) 2024-09-17T21:48:17.989 INFO:teuthology.orchestra.run.smithi165.stdout:5120 write 0x457aa00 thru 0x457e1c1 (0x37c2 bytes) 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 137, 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:48:16.707624+0000", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:47:26.936709+0000", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:48:06.665360+0000", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5806080, 2024-09-17T21:48:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "noscrub", 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 5, 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:48:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:48:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:48:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:48:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:48:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:48:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:48:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 5.880000114440918, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 5.880000114440918, 2024-09-17T21:48:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.17000000178813934, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:48:18.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:48:18.018 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:48:18.019 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:48:18.020 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:48:18.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:48:18.022 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.9900000095367432, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.9900000095367432, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:48:18.024 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:48:18.025 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "137", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 43, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 137, 2024-09-17T21:48:18.026 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:48:18.027 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:48:18.028 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.690000057220459, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.690000057220459, 2024-09-17T21:48:18.029 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.67000001668930054, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.1299999952316284, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.1299999952316284, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:48:18.030 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 131, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.031 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.032 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.033 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:48:18.035 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:48:18.036 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 131, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:48:18.037 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:48:18.038 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:48:18.041 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 0, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.042 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 119, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:48:18.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:48:18.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:48:18.047 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 0, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.048 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 81, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 97, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2260220328 2024-09-17T21:48:18.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/2260220328", 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/2260220328", 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/2260220328", 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/2260220328", 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "exists" 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:48:18.054 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 130, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.055 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.058 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:48:18.060 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0.95001220703125, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 128, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:48:18.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.063 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:48:18.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.066 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:48:18.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:48:18.068 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 62260, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.069 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 97 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:48:18.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-17T21:48:18.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.6", 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "1.0", 2024-09-17T21:48:18.072 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "2.5", 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.073 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.4", 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.5", 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.d", 2024-09-17T21:48:18.074 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.12", 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: 1 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.15", 2024-09-17T21:48:18.075 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1e", 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: 2 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:18.076 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.1f", 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:48:18.077 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [], 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [], 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:48:18.078 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:48:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:48:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:48:18.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:18.079 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:18.081 INFO:tasks.thrashosds.thrasher:j is {'epoch': 137, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:48:16.707624+0000', 'last_up_change': '2024-09-17T21:47:26.936709+0000', 'last_in_change': '2024-09-17T21:48:06.665360+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 5.880000114440918, 'score_stable': 5.880000114440918, 'optimal_score': 0.17000000178813934, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 137, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 119, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 0, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 81, 'down_at': 97, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 2260220328}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 2260220328}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 2260220328}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 2260220328}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 2260220328}]}, 'public_addr': '172.21.15.53:6817/2260220328', 'cluster_addr': '172.21.15.53:6819/2260220328', 'heartbeat_back_addr': '172.21.15.53:6823/2260220328', 'heartbeat_front_addr': '172.21.15.53:6821/2260220328', 'state': ['exists']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 62260, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 97}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '1.0', 'osds': [0, 1]}, {'pgid': '2.5', 'osds': [0, 4]}, {'pgid': '3.4', 'osds': [1, 2]}, {'pgid': '3.5', 'osds': [5, 2]}, {'pgid': '3.d', 'osds': [4, 2]}, {'pgid': '3.12', 'osds': [0, 1]}, {'pgid': '3.15', 'osds': [0, 4]}, {'pgid': '3.1e', 'osds': [1, 2]}, {'pgid': '3.1f', 'osds': [0, 5]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:48:18.081 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-17T21:48:18.082 INFO:teuthology.orchestra.run.smithi165.stdout:5123 write 0xd023600 thru 0xd025b78 (0x2579 bytes) 2024-09-17T21:48:18.082 INFO:teuthology.orchestra.run.smithi165.stdout:5125 trunc from 0xd025c00 to 0x9f05800 2024-09-17T21:48:18.082 INFO:teuthology.orchestra.run.smithi165.stdout:5128 read 0x5483400 thru 0x548bbaa (0x87ab bytes) 2024-09-17T21:48:18.082 INFO:teuthology.orchestra.run.smithi165.stdout:5130 write 0x1b53600 thru 0x1b5882a (0x522b bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5132 read 0x25b7200 thru 0x25c50fb (0xdefc bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5136 write 0x57c9400 thru 0x57d0ecb (0x7acc bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5137 write 0x1d99800 thru 0x1d9d120 (0x3921 bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5139 write 0xc3b4000 thru 0xc3b9fc4 (0x5fc5 bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5140 read 0xd01800 thru 0xd1034c (0xeb4d bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5141 read 0x79d8c00 thru 0x79e5cf9 (0xd0fa bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5144 punch from 0x62d6e00 to 0x62db200, (0x4400 bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5145 write 0x150c000 thru 0x1511452 (0x5453 bytes) 2024-09-17T21:48:18.083 INFO:teuthology.orchestra.run.smithi165.stdout:5146 trunc from 0xc3ba000 to 0x2ba0e00 2024-09-17T21:48:18.083 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:18.077+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:18.085 INFO:teuthology.orchestra.run.smithi165.stdout:5147 punch from 0x57ec00 to 0x586200, (0x7600 bytes) 2024-09-17T21:48:18.095 INFO:teuthology.orchestra.run.smithi165.stdout:5148 read 0x42200 thru 0x44f16 (0x2d17 bytes) 2024-09-17T21:48:18.096 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12243289, 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122410397, 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12238613700001, 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122360254, 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122335253, 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122295454, 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122279709, 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122268174, 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.122257874, 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.120286631, 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12027348300001, 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.120262877, 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12025253900001, 2024-09-17T21:48:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12024253200001, 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.120232581, 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.12021727299999, 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.11824138599999, 2024-09-17T21:48:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.11823043, 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.118217941, 2024-09-17T21:48:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.118207478, 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:18.192 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:18.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5152 write 0x5b1b600 thru 0x5b2a1c1 (0xebc2 bytes) 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5153 write 0x1908200 thru 0x1917693 (0xf494 bytes) 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5154 read 0x3b83000 thru 0x3b8bd88 (0x8d89 bytes) 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5155 punch from 0x4580a00 to 0x4580e00, (0x400 bytes) 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5156 read 0x3e90400 thru 0x3e9d007 (0xcc08 bytes) 2024-09-17T21:48:18.195 INFO:teuthology.orchestra.run.smithi165.stdout:5157 read 0x20e3e00 thru 0x20ec382 (0x8583 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5158 punch from 0x445fa00 to 0x4468e00, (0x9400 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5159 trunc from 0x5b2a200 to 0x4712e00 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5160 punch from 0x4254600 to 0x4258c00, (0x4600 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5163 read 0x4d6200 thru 0x4d8128 (0x1f29 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5165 trunc from 0x4712e00 to 0x99cc600 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5166 write 0x137c000 thru 0x137d0ab (0x10ac bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5167 trunc from 0x99cc600 to 0xc6e5a00 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5169 read 0x7078600 thru 0x7081e86 (0x9887 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5170 punch from 0x5afa000 to 0x5afe600, (0x4600 bytes) 2024-09-17T21:48:18.196 INFO:teuthology.orchestra.run.smithi165.stdout:5173 trunc from 0xc6e5a00 to 0x39af000 2024-09-17T21:48:18.197 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:18.190+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:18.202 INFO:teuthology.orchestra.run.smithi165.stdout:5174 punch from 0x3a5e00 to 0x3b1600, (0xb800 bytes) 2024-09-17T21:48:18.206 INFO:teuthology.orchestra.run.smithi165.stdout:5175 trunc from 0x39af000 to 0x2c13800 2024-09-17T21:48:18.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:18.239 INFO:teuthology.orchestra.run.smithi165.stdout:5176 write 0xe885c00 thru 0xe888809 (0x2c0a bytes) 2024-09-17T21:48:18.239 INFO:teuthology.orchestra.run.smithi165.stdout:5178 write 0xe658a00 thru 0xe65f299 (0x689a bytes) 2024-09-17T21:48:18.240 INFO:teuthology.orchestra.run.smithi165.stdout:5179 write 0x8174800 thru 0x8176a9d (0x229e bytes) 2024-09-17T21:48:18.240 INFO:teuthology.orchestra.run.smithi165.stdout:5182 read 0x13d2600 thru 0x13dc591 (0x9f92 bytes) 2024-09-17T21:48:18.241 INFO:teuthology.orchestra.run.smithi165.stdout:5184 read 0xb71fc00 thru 0xb723060 (0x3461 bytes) 2024-09-17T21:48:18.241 INFO:teuthology.orchestra.run.smithi165.stdout:5185 read 0xdd6aa00 thru 0xdd73e60 (0x9461 bytes) 2024-09-17T21:48:18.241 INFO:teuthology.orchestra.run.smithi165.stdout:5186 punch from 0xaf27a00 to 0xaf32400, (0xaa00 bytes) 2024-09-17T21:48:18.244 INFO:teuthology.orchestra.run.smithi165.stdout:5187 read 0x4631000 thru 0x463d785 (0xc786 bytes) 2024-09-17T21:48:18.252 INFO:teuthology.orchestra.run.smithi165.stdout:5188 trunc from 0xe888a00 to 0x7ec8600 2024-09-17T21:48:18.269 INFO:teuthology.orchestra.run.smithi165.stdout:5189 trunc from 0x7ec8600 to 0x9678800 2024-09-17T21:48:18.278 INFO:teuthology.orchestra.run.smithi165.stdout:5191 punch from 0x1b83e00 to 0x1b8e000, (0xa200 bytes) 2024-09-17T21:48:18.287 INFO:teuthology.orchestra.run.smithi165.stdout:5193 read 0xa75c00 thru 0xa783a5 (0x27a6 bytes) 2024-09-17T21:48:18.288 INFO:teuthology.orchestra.run.smithi165.stdout:5196 read 0x823c800 thru 0x824c335 (0xfb36 bytes) 2024-09-17T21:48:18.288 INFO:teuthology.orchestra.run.smithi165.stdout:5198 read 0x679a600 thru 0x67a4d71 (0xa772 bytes) 2024-09-17T21:48:18.288 INFO:teuthology.orchestra.run.smithi165.stdout:5202 punch from 0x7939c00 to 0x7942000, (0x8400 bytes) 2024-09-17T21:48:18.288 INFO:teuthology.orchestra.run.smithi165.stdout:5203 write 0x493ca00 thru 0x494589f (0x8ea0 bytes) 2024-09-17T21:48:18.289 INFO:teuthology.orchestra.run.smithi165.stdout:5208 read 0x445d600 thru 0x44664ce (0x8ecf bytes) 2024-09-17T21:48:18.289 INFO:teuthology.orchestra.run.smithi165.stdout:5209 read 0x1dace00 thru 0x1daed3d (0x1f3e bytes) 2024-09-17T21:48:18.289 INFO:teuthology.orchestra.run.smithi165.stdout:5211 write 0x6674800 thru 0x6678a49 (0x424a bytes) 2024-09-17T21:48:18.290 INFO:teuthology.orchestra.run.smithi165.stdout:5212 read 0x3fe3c00 thru 0x3fe5cdc (0x20dd bytes) 2024-09-17T21:48:18.293 INFO:teuthology.orchestra.run.smithi165.stdout:5213 trunc from 0x9678800 to 0x69dd400 2024-09-17T21:48:18.294 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:18.291+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:18.313 INFO:teuthology.orchestra.run.smithi165.stdout:5216 trunc from 0x69dd400 to 0x559b600 2024-09-17T21:48:18.317 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:18.318 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:18.318 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:18.318 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:18.325 INFO:teuthology.orchestra.run.smithi165.stdout:5217 punch from 0x1584200 to 0x1592200, (0xe000 bytes) 2024-09-17T21:48:18.328 INFO:teuthology.orchestra.run.smithi165.stdout:5225 punch from 0x512be00 to 0x5131c00, (0x5e00 bytes) 2024-09-17T21:48:18.328 INFO:teuthology.orchestra.run.smithi165.stdout:5227 read 0x2e93a00 thru 0x2e9ab3c (0x713d bytes) 2024-09-17T21:48:18.329 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:18.344 INFO:teuthology.orchestra.run.smithi165.stdout:5231 write 0xb8fba00 thru 0xb8ffd09 (0x430a bytes) 2024-09-17T21:48:18.345 INFO:teuthology.orchestra.run.smithi165.stdout:5232 punch from 0x29d8000 to 0x29df400, (0x7400 bytes) 2024-09-17T21:48:18.348 INFO:teuthology.orchestra.run.smithi165.stdout:5233 read 0x6778400 thru 0x6783624 (0xb225 bytes) 2024-09-17T21:48:18.349 INFO:teuthology.orchestra.run.smithi165.stdout:5236 trunc from 0xb8ffe00 to 0xce11600 2024-09-17T21:48:18.360 INFO:teuthology.orchestra.run.smithi165.stdout:5238 punch from 0x74b0a00 to 0x74b8200, (0x7800 bytes) 2024-09-17T21:48:18.369 INFO:teuthology.orchestra.run.smithi165.stdout:5240 write 0x8854200 thru 0x885b3bb (0x71bc bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5241 punch from 0x2c29a00 to 0x2c2b200, (0x1800 bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5243 punch from 0x5a72a00 to 0x5a72c00, (0x200 bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5244 write 0x35ac600 thru 0x35adfce (0x19cf bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5245 read 0x390200 thru 0x39cce7 (0xcae8 bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5246 write 0xbcdc800 thru 0xbce31cf (0x69d0 bytes) 2024-09-17T21:48:18.370 INFO:teuthology.orchestra.run.smithi165.stdout:5251 write 0xc416000 thru 0xc418f9f (0x2fa0 bytes) 2024-09-17T21:48:18.371 INFO:teuthology.orchestra.run.smithi165.stdout:5252 trunc from 0xce11600 to 0xd79ee00 2024-09-17T21:48:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:18.390 INFO:teuthology.orchestra.run.smithi165.stdout:5254 read 0xa7f5a00 thru 0xa7f91dc (0x37dd bytes) 2024-09-17T21:48:18.392 INFO:teuthology.orchestra.run.smithi165.stdout:5255 write 0xc055600 thru 0xc063479 (0xde7a bytes) 2024-09-17T21:48:18.392 INFO:teuthology.orchestra.run.smithi165.stdout:5257 read 0x6c2a400 thru 0x6c2b2a6 (0xea7 bytes) 2024-09-17T21:48:18.392 INFO:teuthology.orchestra.run.smithi165.stdout:5259 write 0xccbf400 thru 0xcccb34c (0xbf4d bytes) 2024-09-17T21:48:18.394 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:18.392+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:18.400 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:18.406 INFO:teuthology.orchestra.run.smithi165.stdout:5260 trunc from 0xd79ee00 to 0x1f60400 2024-09-17T21:48:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:18.420 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:18.431 INFO:teuthology.orchestra.run.smithi165.stdout:5261 read 0x1c6a800 thru 0x1c7031e (0x5b1f bytes) 2024-09-17T21:48:18.433 INFO:teuthology.orchestra.run.smithi165.stdout:5262 punch from 0x10c6c00 to 0x10cd800, (0x6c00 bytes) 2024-09-17T21:48:18.435 INFO:teuthology.orchestra.run.smithi165.stdout:5263 read 0x228800 thru 0x2310e7 (0x88e8 bytes) 2024-09-17T21:48:18.435 INFO:teuthology.orchestra.run.smithi165.stdout:5264 read 0x12b5600 thru 0x12ba33b (0x4d3c bytes) 2024-09-17T21:48:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:18.441 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:18.448 INFO:teuthology.orchestra.run.smithi165.stdout:5266 write 0x406b200 thru 0x4077f09 (0xcd0a bytes) 2024-09-17T21:48:18.449 INFO:teuthology.orchestra.run.smithi165.stdout:5267 clone 42 order 19 su 65536 sc 13 2024-09-17T21:48:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:18.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:18.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:18.464 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:18.495 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:18.493+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:18.596 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:18.594+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:18.610 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:18.611 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:18.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:18.697 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:18.694+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:18.717 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:48:18.770 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:18.770 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:18.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:18.797 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:18.795+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:18.803 INFO:teuthology.orchestra.run.smithi165.stdout:flattening image image_client.0-clone41 2024-09-17T21:48:18.899 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:18.896+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:18.928 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:18.928 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:18.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:19.000 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:18.998+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:19.046 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.046 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:19.101 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:19.099+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:19.124 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-17T21:48:19.202 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.203 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.203 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:19.200+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:19.205 INFO:teuthology.orchestra.run.smithi165.stdout:5270 punch from 0x25e7c00 to 0x25f7000, (0xf400 bytes) 2024-09-17T21:48:19.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:19.214 INFO:teuthology.orchestra.run.smithi165.stdout:5271 read 0xdd3a00 thru 0xddadfc (0x73fd bytes) 2024-09-17T21:48:19.214 INFO:teuthology.orchestra.run.smithi165.stdout:5272 read 0x3cd4c00 thru 0x3cd7117 (0x2518 bytes) 2024-09-17T21:48:19.228 INFO:teuthology.orchestra.run.smithi165.stdout:5274 write 0x8338600 thru 0x8339ba8 (0x15a9 bytes) 2024-09-17T21:48:19.234 INFO:teuthology.orchestra.run.smithi165.stdout:5275 write 0x6da200 thru 0x6e0306 (0x6107 bytes) 2024-09-17T21:48:19.249 INFO:teuthology.orchestra.run.smithi165.stdout:5276 write 0x8747a00 thru 0x8755790 (0xdd91 bytes) 2024-09-17T21:48:19.260 INFO:teuthology.orchestra.run.smithi165.stdout:5278 trunc from 0x8755800 to 0x3eff000 2024-09-17T21:48:19.300 INFO:teuthology.orchestra.run.smithi165.stdout:5279 write 0x4648400 thru 0x465303d (0xac3e bytes) 2024-09-17T21:48:19.303 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:19.301+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:19.318 INFO:teuthology.orchestra.run.smithi165.stdout:5283 write 0x469b800 thru 0x469ea52 (0x3253 bytes) 2024-09-17T21:48:19.319 INFO:teuthology.orchestra.run.smithi165.stdout:5284 read 0x254ba00 thru 0x255a6fc (0xecfd bytes) 2024-09-17T21:48:19.319 INFO:teuthology.orchestra.run.smithi165.stdout:5285 read 0x22ec600 thru 0x22ef40b (0x2e0c bytes) 2024-09-17T21:48:19.333 INFO:teuthology.orchestra.run.smithi165.stdout:5287 write 0x679d400 thru 0x679e05b (0xc5c bytes) 2024-09-17T21:48:19.334 INFO:teuthology.orchestra.run.smithi165.stdout:5289 write 0x4bb2200 thru 0x4bb3e06 (0x1c07 bytes) 2024-09-17T21:48:19.334 INFO:teuthology.orchestra.run.smithi165.stdout:5291 read 0x41b8800 thru 0x41bf04c (0x684d bytes) 2024-09-17T21:48:19.335 INFO:teuthology.orchestra.run.smithi165.stdout:5293 read 0x1103a00 thru 0x111114c (0xd74d bytes) 2024-09-17T21:48:19.335 INFO:teuthology.orchestra.run.smithi165.stdout:5297 write 0x3f02e00 thru 0x3f0b316 (0x8517 bytes) 2024-09-17T21:48:19.341 INFO:teuthology.orchestra.run.smithi165.stdout:5299 trunc from 0x679e200 to 0x2199800 2024-09-17T21:48:19.368 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.368 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:19.383 INFO:teuthology.orchestra.run.smithi165.stdout:5301 write 0xe73200 thru 0xe73ea2 (0xca3 bytes) 2024-09-17T21:48:19.397 INFO:teuthology.orchestra.run.smithi165.stdout:5302 write 0x3783e00 thru 0x37871df (0x33e0 bytes) 2024-09-17T21:48:19.398 INFO:teuthology.orchestra.run.smithi165.stdout:5303 punch from 0x2868e00 to 0x286f000, (0x6200 bytes) 2024-09-17T21:48:19.418 INFO:teuthology.orchestra.run.smithi165.stdout:5304 write 0xd46e600 thru 0xd47a2f6 (0xbcf7 bytes) 2024-09-17T21:48:19.419 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:19.417+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:19.435 INFO:teuthology.orchestra.run.smithi165.stdout:5307 write 0xe927600 thru 0xe9348aa (0xd2ab bytes) 2024-09-17T21:48:19.440 INFO:teuthology.orchestra.run.smithi165.stdout:5308 punch from 0xb28be00 to 0xb294a00, (0x8c00 bytes) 2024-09-17T21:48:19.442 INFO:teuthology.orchestra.run.smithi165.stdout:5310 write 0xbaa2000 thru 0xbaabe21 (0x9e22 bytes) 2024-09-17T21:48:19.442 INFO:teuthology.orchestra.run.smithi165.stdout:5311 read 0x6b58200 thru 0x6b59343 (0x1144 bytes) 2024-09-17T21:48:19.443 INFO:teuthology.orchestra.run.smithi165.stdout:5313 read 0x710ba00 thru 0x711a09a (0xe69b bytes) 2024-09-17T21:48:19.443 INFO:teuthology.orchestra.run.smithi165.stdout:5315 read 0xd1da200 thru 0xd1e1eef (0x7cf0 bytes) 2024-09-17T21:48:19.443 INFO:teuthology.orchestra.run.smithi165.stdout:5316 read 0x460f000 thru 0x461ac5a (0xbc5b bytes) 2024-09-17T21:48:19.443 INFO:teuthology.orchestra.run.smithi165.stdout:5317 punch from 0xe285200 to 0xe28fa00, (0xa800 bytes) 2024-09-17T21:48:19.443 INFO:teuthology.orchestra.run.smithi165.stdout:5319 read 0xdb6000 thru 0xdc2878 (0xc879 bytes) 2024-09-17T21:48:19.444 INFO:teuthology.orchestra.run.smithi165.stdout:5320 read 0x311be00 thru 0x3126c7b (0xae7c bytes) 2024-09-17T21:48:19.444 INFO:teuthology.orchestra.run.smithi165.stdout:5323 read 0x7dcfa00 thru 0x7dde6a2 (0xeca3 bytes) 2024-09-17T21:48:19.444 INFO:teuthology.orchestra.run.smithi165.stdout:5324 punch from 0x47cce00 to 0x47d2c00, (0x5e00 bytes) 2024-09-17T21:48:19.444 INFO:teuthology.orchestra.run.smithi165.stdout:5325 read 0x37400 thru 0x3d9f8 (0x65f9 bytes) 2024-09-17T21:48:19.445 INFO:teuthology.orchestra.run.smithi165.stdout:5326 punch from 0x10a5000 to 0x10aec00, (0x9c00 bytes) 2024-09-17T21:48:19.445 INFO:teuthology.orchestra.run.smithi165.stdout:5331 write 0x9c28e00 thru 0x9c2dccd (0x4ece bytes) 2024-09-17T21:48:19.452 INFO:teuthology.orchestra.run.smithi165.stdout:5332 trunc from 0xe934a00 to 0x87d1a00 2024-09-17T21:48:19.470 INFO:teuthology.orchestra.run.smithi165.stdout:5333 clone 43 order 18 su 131072 sc 4 2024-09-17T21:48:19.505 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:19.502+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:19.530 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.530 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.539 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:19.606 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:19.604+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:19.646 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.646 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.655 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:19.707 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:19.704+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:19.765 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.765 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:19.792 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone42 from 0x87d1a00 (overlap 0x2199800) to 0xd70800 2024-09-17T21:48:19.808 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:19.805+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:19.905 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:19.905 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:19.909 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:19.906+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:19.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:20.009 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:20.007+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:20.064 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:20.065 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:20.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:20.111 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:20.108+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:20.111 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-17T21:48:20.186 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:20.186 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:20.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:20.239 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:20.237+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:20.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:20.308 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:20.312 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:20.309+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:20.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:20.353 INFO:teuthology.orchestra.run.smithi165.stdout:5334 write 0x7e0f800 thru 0x7e1a03d (0xa83e bytes) 2024-09-17T21:48:20.380 INFO:teuthology.orchestra.run.smithi165.stdout:5336 write 0xebc7a00 thru 0xebd781d (0xfe1e bytes) 2024-09-17T21:48:20.381 INFO:teuthology.orchestra.run.smithi165.stdout:5338 read 0x294f000 thru 0x2958a35 (0x9a36 bytes) 2024-09-17T21:48:20.381 INFO:teuthology.orchestra.run.smithi165.stdout:5339 punch from 0xcca6800 to 0xccaf400, (0x8c00 bytes) 2024-09-17T21:48:20.383 INFO:teuthology.orchestra.run.smithi165.stdout:5340 punch from 0x3011800 to 0x3020400, (0xec00 bytes) 2024-09-17T21:48:20.383 INFO:teuthology.orchestra.run.smithi165.stdout:5341 punch from 0xe1bae00 to 0xe1c2c00, (0x7e00 bytes) 2024-09-17T21:48:20.384 INFO:teuthology.orchestra.run.smithi165.stdout:5342 write 0xa143e00 thru 0xa15116b (0xd36c bytes) 2024-09-17T21:48:20.384 INFO:teuthology.orchestra.run.smithi165.stdout:5343 read 0x404d200 thru 0x405150a (0x430b bytes) 2024-09-17T21:48:20.384 INFO:teuthology.orchestra.run.smithi165.stdout:5344 write 0xd4d3400 thru 0xd4d5a33 (0x2634 bytes) 2024-09-17T21:48:20.384 INFO:teuthology.orchestra.run.smithi165.stdout:5350 read 0x1ad5000 thru 0x1ad90b3 (0x40b4 bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5352 read 0x557bc00 thru 0x55821c5 (0x65c6 bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5353 write 0x2724a00 thru 0x272ada7 (0x63a8 bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5354 read 0x3dee200 thru 0x3dfbcea (0xdaeb bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5356 write 0xe3ae600 thru 0xe3b14dd (0x2ede bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5357 read 0xd353400 thru 0xd355373 (0x1f74 bytes) 2024-09-17T21:48:20.385 INFO:teuthology.orchestra.run.smithi165.stdout:5358 read 0x9d95400 thru 0x9d9b14c (0x5d4d bytes) 2024-09-17T21:48:20.387 INFO:teuthology.orchestra.run.smithi165.stdout:5361 trunc from 0xebd7a00 to 0xcad4400 2024-09-17T21:48:20.413 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:20.411+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:20.416 INFO:teuthology.orchestra.run.smithi165.stdout:5362 trunc from 0xcad4400 to 0x5dc7000 2024-09-17T21:48:20.439 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:20.439 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:20.440 INFO:teuthology.orchestra.run.smithi165.stdout:5365 trunc from 0x5dc7000 to 0x8fed800 2024-09-17T21:48:20.448 INFO:teuthology.orchestra.run.smithi165.stdout:5366 punch from 0x71f8200 to 0x71fd600, (0x5400 bytes) 2024-09-17T21:48:20.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:20.451 INFO:teuthology.orchestra.run.smithi165.stdout:5369 punch from 0x782b600 to 0x7834000, (0x8a00 bytes) 2024-09-17T21:48:20.451 INFO:teuthology.orchestra.run.smithi165.stdout:5370 write 0x2f2da00 thru 0x2f3bde5 (0xe3e6 bytes) 2024-09-17T21:48:20.451 INFO:teuthology.orchestra.run.smithi165.stdout:5373 read 0x2b4be00 thru 0x2b56f54 (0xb155 bytes) 2024-09-17T21:48:20.465 INFO:teuthology.orchestra.run.smithi165.stdout:5376 write 0xaa30000 thru 0xaa30b3b (0xb3c bytes) 2024-09-17T21:48:20.465 INFO:teuthology.orchestra.run.smithi165.stdout:5377 read 0x7bb9600 thru 0x7bbfbf5 (0x65f6 bytes) 2024-09-17T21:48:20.465 INFO:teuthology.orchestra.run.smithi165.stdout:5378 write 0xab1000 thru 0xabca08 (0xba09 bytes) 2024-09-17T21:48:20.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:20.466 INFO:teuthology.orchestra.run.smithi165.stdout:5380 write 0x416c800 thru 0x41788ae (0xc0af bytes) 2024-09-17T21:48:20.466 INFO:teuthology.orchestra.run.smithi165.stdout:5381 write 0x7715800 thru 0x7720a64 (0xb265 bytes) 2024-09-17T21:48:20.467 INFO:teuthology.orchestra.run.smithi165.stdout:5382 read 0x7cd8c00 thru 0x7cdafcf (0x23d0 bytes) 2024-09-17T21:48:20.467 INFO:teuthology.orchestra.run.smithi165.stdout:5383 punch from 0x8725c00 to 0x8728a00, (0x2e00 bytes) 2024-09-17T21:48:20.473 INFO:teuthology.orchestra.run.smithi165.stdout:5384 trunc from 0xaa30c00 to 0x5337a00 2024-09-17T21:48:20.492 INFO:teuthology.orchestra.run.smithi165.stdout:5385 read 0x3c53a00 thru 0x3c5a05b (0x665c bytes) 2024-09-17T21:48:20.498 INFO:teuthology.orchestra.run.smithi165.stdout:5387 trunc from 0x5337a00 to 0x9092600 2024-09-17T21:48:20.508 INFO:teuthology.orchestra.run.smithi165.stdout:5388 punch from 0x6cfbe00 to 0x6d01000, (0x5200 bytes) 2024-09-17T21:48:20.516 INFO:teuthology.orchestra.run.smithi165.stdout:5389 trunc from 0x9092600 to 0x7c16c00 2024-09-17T21:48:20.528 INFO:teuthology.orchestra.run.smithi165.stdout:5390 punch from 0x1fd6200 to 0x1fd6a00, (0x800 bytes) 2024-09-17T21:48:20.531 INFO:teuthology.orchestra.run.smithi165.stdout:5391 write 0xe09400 thru 0xe0ceb2 (0x3ab3 bytes) 2024-09-17T21:48:20.532 INFO:teuthology.orchestra.run.smithi165.stdout:5392 write 0x7138600 thru 0x71458c2 (0xd2c3 bytes) 2024-09-17T21:48:20.533 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:20.531+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:20.551 INFO:teuthology.orchestra.run.smithi165.stdout:5394 write 0xa1db200 thru 0xa1e96ea (0xe4eb bytes) 2024-09-17T21:48:20.553 INFO:teuthology.orchestra.run.smithi165.stdout:5395 write 0x8c10a00 thru 0x8c1ff85 (0xf586 bytes) 2024-09-17T21:48:20.553 INFO:teuthology.orchestra.run.smithi165.stdout:5396 read 0x63ef800 thru 0x63efdf4 (0x5f5 bytes) 2024-09-17T21:48:20.563 INFO:teuthology.orchestra.run.smithi165.stdout:5399 trunc from 0xa1e9800 to 0x107c400 2024-09-17T21:48:20.603 INFO:teuthology.orchestra.run.smithi165.stdout:5400 trunc from 0x107c400 to 0x56eb200 2024-09-17T21:48:20.615 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:20.612+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:20.617 INFO:teuthology.orchestra.run.smithi165.stdout:5402 punch from 0x46e1400 to 0x46e6a00, (0x5600 bytes) 2024-09-17T21:48:20.622 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:20.624 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:20.625 INFO:teuthology.orchestra.run.smithi165.stdout:5403 clone 44 order 18 su 16384 sc 11 2024-09-17T21:48:20.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:20.639 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:20.639 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:20.656 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:20.656 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:20.715 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:20.713+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:20.733 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.067298178, 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:20.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:20.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:20.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:20.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.03502320299999, 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:20.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:20.765 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.03478443500001, 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:20.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:20.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.867278304999999, 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:20.771 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:20.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:20.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:20.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.860376533999997, 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.777 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.860260443000001, 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:20.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:20.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.663256922999999, 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:20.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.663218127999997, 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.793 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:20.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:20.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.663203844000002, 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:20.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:20.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:20.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.662958877999998, 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:20.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:20.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:20.807 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.662876429999997, 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:20.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:20.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.662844620000001, 2024-09-17T21:48:20.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:20.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:20.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.662820901000003, 2024-09-17T21:48:20.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:20.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:20.822 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.276181504, 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:20.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 31.266252523999999, 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:20.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:20.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:20.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.073917809999999, 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.834 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:20.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:20.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.073660389, 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:20.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:20.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:20.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.073627543000001, 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:20.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:20.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:20.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.073610598, 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.853 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:20.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:20.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:20.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.073569101, 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:20.859 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:20.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:20.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:20.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:20.865 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone43 intact 2024-09-17T21:48:20.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:20.867 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:20.863+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:20.873 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:20.874 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:20.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:20.893 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:20.913 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:20.913 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:20.916 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:20.917 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:20.966 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:20.964+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:20.986 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:20.986 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:20.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:21.067 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:21.064+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:21.145 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-17T21:48:21.146 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:21.146 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:21.146 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:21.146 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:21.146 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:21.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:21.200 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:21.198+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:21.269 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:21.266+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:21.273 INFO:teuthology.orchestra.run.smithi165.stdout:5404 read 0x41cb800 thru 0x41db518 (0xfd19 bytes) 2024-09-17T21:48:21.274 INFO:teuthology.orchestra.run.smithi165.stdout:5408 read 0x3f75800 thru 0x3f7e83e (0x903f bytes) 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.188817174, 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:21.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187674886, 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187658841000001, 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187642296, 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187627531, 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187600226000001, 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187563338, 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187524639999999, 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187486466999999, 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187449038, 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187392015, 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187361344999999, 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187216972000002, 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187198126999998, 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187171531000001, 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.187136691999999, 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.612633724999998, 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.612600409999999, 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.605510345999999, 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.605476666000001, 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.378 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:21.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5409 trunc from 0x56eb200 to 0x770fa00 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5410 read 0x4efda00 thru 0x4f008d5 (0x2ed6 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5411 read 0x7222000 thru 0x722c1a3 (0xa1a4 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5412 punch from 0x2d61c00 to 0x2d66c00, (0x5000 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5413 punch from 0x5999c00 to 0x59a6600, (0xca00 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5415 punch from 0x505fc00 to 0x506aa00, (0xae00 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5419 trunc from 0x770fa00 to 0x3363c00 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5420 write 0x9fe3400 thru 0x9fe9e32 (0x6a33 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5422 read 0x6811200 thru 0x6816a38 (0x5839 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5423 write 0x5cf6e00 thru 0x5cf79a0 (0xba1 bytes) 2024-09-17T21:48:21.381 INFO:teuthology.orchestra.run.smithi165.stdout:5426 clone 45 order 20 su 262144 sc 14 2024-09-17T21:48:21.382 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:21.376+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:21.480 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:21.478+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:21.528 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:21.528 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:21.528 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:21.528 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:21.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:21.581 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:21.579+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:21.681 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:21.679+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:21.685 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:21.685 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:21.685 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:21.686 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:21.686 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:21.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:21.730 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:48:21.782 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:21.780+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.810 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 106.934435284, 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:21.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:21.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:21.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897988974, 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:21.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.821 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897967672999997, 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:21.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897946912000002, 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:21.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897925806000003, 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:21.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897905317000003, 2024-09-17T21:48:21.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:21.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897881220000002, 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.840 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:21.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897858726999999, 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:21.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:21.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897831119999999, 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:21.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 34.897806279999998, 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:21.854 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:21.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.858 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.745852504999998, 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.859 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.863 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.745828743000001, 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:21.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.745786354, 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:21.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:21.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.690037701000001, 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:21.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:21.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.690014605999998, 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:21.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.689988226000001, 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.689971589999999, 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.689941626, 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.689928700999999, 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 30.689918479999999, 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:21.904 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:21.905 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone44 from 0x9fea000 (overlap 0x3363c00) to 0xf6aa00 2024-09-17T21:48:21.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:21.906 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:21.903+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:22.006 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:22.003+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:22.014 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.014 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:22.014 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:22.014 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:22.106 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:22.104+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:22.130 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:22.130 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.139 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:22.208 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:22.205+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.50809440800001, 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:22.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:22.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.507049948, 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:22.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.305864616999997, 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:22.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:22.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:22.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.089946949000002, 2024-09-17T21:48:22.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:22.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.887510206999998, 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:22.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:22.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:22.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.887453026999999, 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:22.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:22.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886882446000001, 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:22.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:22.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:22.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886854511999999, 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:22.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:22.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886720159999999, 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:22.298 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:22.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886698625000001, 2024-09-17T21:48:22.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:22.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:22.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:22.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886537659999998, 2024-09-17T21:48:22.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:22.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:22.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.886481894999999, 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:22.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.885135288000001, 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:22.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:22.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.885083731999998, 2024-09-17T21:48:22.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:22.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.884395471000001, 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:22.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.884378420000001, 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.335 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:22.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:22.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.883440180000001, 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:22.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:22.345 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.883409839999999, 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:22.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.589012392000001, 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:22.351 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:22.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.687894692, 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:22.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.362 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.363 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-17T21:48:22.364 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:22.365 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:22.361+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:22.464 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:22.461+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:22.517 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:22.517 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:22.517 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:22.565 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:22.563+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:22.573 INFO:teuthology.orchestra.run.smithi165.stdout:5430 read 0x4d5ec00 thru 0x4d6aa03 (0xbe04 bytes) 2024-09-17T21:48:22.573 INFO:teuthology.orchestra.run.smithi165.stdout:5431 read 0x581e800 thru 0x58208d7 (0x20d8 bytes) 2024-09-17T21:48:22.574 INFO:teuthology.orchestra.run.smithi165.stdout:5434 read 0x671f400 thru 0x672674d (0x734e bytes) 2024-09-17T21:48:22.594 INFO:teuthology.orchestra.run.smithi165.stdout:5437 write 0xe6a8e00 thru 0xe6abcd4 (0x2ed5 bytes) 2024-09-17T21:48:22.600 INFO:teuthology.orchestra.run.smithi165.stdout:5439 punch from 0xc0dea00 to 0xc0e5e00, (0x7400 bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5440 write 0x75cd000 thru 0x75d5c0d (0x8c0e bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5441 read 0x6494400 thru 0x649ad6e (0x696f bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5444 write 0x7d71000 thru 0x7d779ec (0x69ed bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5445 read 0x5931600 thru 0x593a1ce (0x8bcf bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5448 punch from 0x872200 to 0x87ae00, (0x8c00 bytes) 2024-09-17T21:48:22.601 INFO:teuthology.orchestra.run.smithi165.stdout:5451 read 0x9f75e00 thru 0x9f778e0 (0x1ae1 bytes) 2024-09-17T21:48:22.602 INFO:teuthology.orchestra.run.smithi165.stdout:5452 write 0x199a200 thru 0x19a845f (0xe260 bytes) 2024-09-17T21:48:22.602 INFO:teuthology.orchestra.run.smithi165.stdout:5453 read 0x94b600 thru 0x951293 (0x5c94 bytes) 2024-09-17T21:48:22.604 INFO:teuthology.orchestra.run.smithi165.stdout:5454 punch from 0x65d4a00 to 0x65e0a00, (0xc000 bytes) 2024-09-17T21:48:22.605 INFO:teuthology.orchestra.run.smithi165.stdout:5455 write 0x7482c00 thru 0x748b54d (0x894e bytes) 2024-09-17T21:48:22.605 INFO:teuthology.orchestra.run.smithi165.stdout:5457 write 0x57dc600 thru 0x57e3266 (0x6c67 bytes) 2024-09-17T21:48:22.605 INFO:teuthology.orchestra.run.smithi165.stdout:5459 write 0xdbf7400 thru 0xdbfd72d (0x632e bytes) 2024-09-17T21:48:22.605 INFO:teuthology.orchestra.run.smithi165.stdout:5461 read 0x972c200 thru 0x9738d29 (0xcb2a bytes) 2024-09-17T21:48:22.605 INFO:teuthology.orchestra.run.smithi165.stdout:5463 read 0xb572e00 thru 0xb57f682 (0xc883 bytes) 2024-09-17T21:48:22.606 INFO:teuthology.orchestra.run.smithi165.stdout:5464 read 0x76da400 thru 0x76e3b05 (0x9706 bytes) 2024-09-17T21:48:22.606 INFO:teuthology.orchestra.run.smithi165.stdout:5467 write 0xcd67600 thru 0xcd6fa53 (0x8454 bytes) 2024-09-17T21:48:22.606 INFO:teuthology.orchestra.run.smithi165.stdout:5469 write 0x2330c00 thru 0x233da2e (0xce2f bytes) 2024-09-17T21:48:22.606 INFO:teuthology.orchestra.run.smithi165.stdout:5470 write 0x8eb2e00 thru 0x8eb84ef (0x56f0 bytes) 2024-09-17T21:48:22.606 INFO:teuthology.orchestra.run.smithi165.stdout:5471 write 0xb373200 thru 0xb375d2a (0x2b2b bytes) 2024-09-17T21:48:22.611 INFO:teuthology.orchestra.run.smithi165.stdout:5473 trunc from 0xe6abe00 to 0xaa6da00 2024-09-17T21:48:22.635 INFO:teuthology.orchestra.run.smithi165.stdout:5476 read 0x16d4000 thru 0x16d79bb (0x39bc bytes) 2024-09-17T21:48:22.641 INFO:teuthology.orchestra.run.smithi165.stdout:5477 punch from 0x411a000 to 0x4125e00, (0xbe00 bytes) 2024-09-17T21:48:22.642 INFO:teuthology.orchestra.run.smithi165.stdout:5478 read 0x883ea00 thru 0x884c86f (0xde70 bytes) 2024-09-17T21:48:22.643 INFO:teuthology.orchestra.run.smithi165.stdout:5479 write 0x5e25000 thru 0x5e317cc (0xc7cd bytes) 2024-09-17T21:48:22.643 INFO:teuthology.orchestra.run.smithi165.stdout:5480 read 0x5de1400 thru 0x5de3988 (0x2589 bytes) 2024-09-17T21:48:22.643 INFO:teuthology.orchestra.run.smithi165.stdout:5482 write 0x6f1c00 thru 0x6fa88f (0x8c90 bytes) 2024-09-17T21:48:22.644 INFO:teuthology.orchestra.run.smithi165.stdout:5483 read 0xaa09800 thru 0xaa0a394 (0xb95 bytes) 2024-09-17T21:48:22.644 INFO:teuthology.orchestra.run.smithi165.stdout:5484 write 0x9425000 thru 0x942d0c5 (0x80c6 bytes) 2024-09-17T21:48:22.644 INFO:teuthology.orchestra.run.smithi165.stdout:5485 read 0x33ea600 thru 0x33eeadd (0x44de bytes) 2024-09-17T21:48:22.645 INFO:teuthology.orchestra.run.smithi165.stdout:5488 read 0x9291000 thru 0x92929c1 (0x19c2 bytes) 2024-09-17T21:48:22.645 INFO:teuthology.orchestra.run.smithi165.stdout:5491 read 0x30ec400 thru 0x30f7db3 (0xb9b4 bytes) 2024-09-17T21:48:22.645 INFO:teuthology.orchestra.run.smithi165.stdout:5496 read 0x6c5e00 thru 0x6c7783 (0x1984 bytes) 2024-09-17T21:48:22.645 INFO:teuthology.orchestra.run.smithi165.stdout:5497 write 0x19d5200 thru 0x19dc091 (0x6e92 bytes) 2024-09-17T21:48:22.647 INFO:teuthology.orchestra.run.smithi165.stdout:5499 trunc from 0xaa6da00 to 0x8f20c00 2024-09-17T21:48:22.666 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:22.664+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:22.684 INFO:teuthology.orchestra.run.smithi165.stdout:5501 clone 46 order 18 su 65536 sc 8 2024-09-17T21:48:22.693 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:22.693 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:22.693 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.702 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:22.767 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:22.764+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:22.819 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:22.819 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845424095, 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:22.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845401602, 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:22.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.84537734200001, 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:22.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845351459, 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:22.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:22.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845326458, 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:22.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:22.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845286659, 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:22.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845270914, 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:22.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845259379, 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:22.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:22.858 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.845249079, 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:22.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:22.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.843277836, 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.843264688, 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:22.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:22.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.843254082, 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:22.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:22.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.84324374400001, 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:22.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.84323373700001, 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:22.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.843223786, 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:22.888 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:22.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.84320847799999, 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:22.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.84123259099999, 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:22.898 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:22.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.841221635, 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:22.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.841209146, 2024-09-17T21:48:22.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:22.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 125.841198683, 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:22.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:22.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:22.916 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:22.917 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone45 from 0x8f20c00 (overlap 0x8f20c00) to 0x2af0200 2024-09-17T21:48:22.918 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:22.915+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:22.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:22.920 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:23.018 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:23.016+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:23.031 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:23.031 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:23.031 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:23.032 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:23.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:23.053 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:23.053 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:23.070 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:23.070 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:23.082 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 4, 0] out_osds: [2] dead_osds: [3] live_osds: [0, 1, 2, 4, 5] 2024-09-17T21:48:23.082 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:23.082 INFO:tasks.thrashosds.thrasher:Reviving osd 3 2024-09-17T21:48:23.082 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T21:48:23.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T21:48:23.085 INFO:tasks.ceph.osd.3:Started 2024-09-17T21:48:23.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:23.088 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:23.088 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:23.088 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:23.121 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:23.121 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:23.128 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:23.126+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:23.139 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:23.139 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:23.142 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:23.143 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:23.175 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.175 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.185 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:config diff 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:config diff get 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:config get 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:config help [] 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:config set ... 2024-09-17T21:48:23.194 INFO:teuthology.orchestra.run.smithi053.stderr:admin_socket: invalid command 2024-09-17T21:48:23.196 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:48:23.196 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-17T21:48:23.238 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:23.236+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:23.292 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-17T21:48:23.321 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:23.319+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:23.352 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.352 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:23.449 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:23.447+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:23.511 INFO:teuthology.orchestra.run.smithi165.stdout:5502 trunc from 0x8f20c00 to 0x1114000 2024-09-17T21:48:23.520 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.520 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.522 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:23.520+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:23.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:23.537 INFO:teuthology.orchestra.run.smithi165.stdout:5506 punch from 0x3d2c00 to 0x3de600, (0xba00 bytes) 2024-09-17T21:48:23.565 INFO:teuthology.orchestra.run.smithi165.stdout:5508 write 0xaa62800 thru 0xaa67303 (0x4b04 bytes) 2024-09-17T21:48:23.571 INFO:teuthology.orchestra.run.smithi165.stdout:5510 punch from 0x903c00 to 0x907800, (0x3c00 bytes) 2024-09-17T21:48:23.573 INFO:teuthology.orchestra.run.smithi165.stdout:5511 clone 47 order 24 su 524288 sc 12 2024-09-17T21:48:23.623 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:23.621+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:23.680 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.680 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:23.732 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:23.730+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:23.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:23.800 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.801 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:23.847 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone46 from 0xaa67400 (overlap 0x1114000) to 0xda9800 2024-09-17T21:48:23.854 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:23.851+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:23.926 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:23.923+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:23.932 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:23.932 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:23.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:24.027 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:24.025+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:24.091 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.091 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.100 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:24.145 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:24.143+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:24.212 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.212 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.221 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:24.262 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:24.260+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:24.297 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-09-17T21:48:24.329 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:24.326+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:24.332 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.332 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:24.431 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:24.428+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:24.502 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.502 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.509 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:24.506+0000 7fcf3053e640 -1 Falling back to public interface 2024-09-17T21:48:24.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:24.532 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:24.529+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:24.633 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:24.630+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:24.653 INFO:teuthology.orchestra.run.smithi165.stdout:5512 write 0xbba1e00 thru 0xbba9f17 (0x8118 bytes) 2024-09-17T21:48:24.653 INFO:teuthology.orchestra.run.smithi165.stdout:5513 read 0x3a3e800 thru 0x3a48954 (0xa155 bytes) 2024-09-17T21:48:24.653 INFO:teuthology.orchestra.run.smithi165.stdout:5516 read 0x3323000 thru 0x332a4ac (0x74ad bytes) 2024-09-17T21:48:24.653 INFO:teuthology.orchestra.run.smithi165.stdout:5517 punch from 0x9f6ce00 to 0x9f6ec00, (0x1e00 bytes) 2024-09-17T21:48:24.662 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.662 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.672 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:24.732 INFO:teuthology.orchestra.run.smithi165.stdout:5519 write 0xbe12600 thru 0xbe18906 (0x6307 bytes) 2024-09-17T21:48:24.733 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:24.731+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:24.738 INFO:teuthology.orchestra.run.smithi165.stdout:5520 trunc from 0xbe18a00 to 0x6f93000 2024-09-17T21:48:24.742 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:48:24.790 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.790 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.802 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:24.824 INFO:teuthology.orchestra.run.smithi165.stdout:5521 trunc from 0x6f93000 to 0x55c3e00 2024-09-17T21:48:24.843 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:24.841+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:24.844 INFO:teuthology.orchestra.run.smithi165.stdout:5522 read 0xe7e600 thru 0xe87a4e (0x944f bytes) 2024-09-17T21:48:24.844 INFO:teuthology.orchestra.run.smithi165.stdout:5523 punch from 0x382ee00 to 0x383e600, (0xf800 bytes) 2024-09-17T21:48:24.844 INFO:teuthology.orchestra.run.smithi165.stdout:5525 read 0x739800 thru 0x73c805 (0x3006 bytes) 2024-09-17T21:48:24.861 INFO:teuthology.orchestra.run.smithi165.stdout:5528 write 0xad39e00 thru 0xad3ea83 (0x4c84 bytes) 2024-09-17T21:48:24.875 INFO:teuthology.orchestra.run.smithi165.stdout:5531 write 0xcb2b200 thru 0xcb2b3ce (0x1cf bytes) 2024-09-17T21:48:24.875 INFO:teuthology.orchestra.run.smithi165.stdout:5532 clone 48 order 25 su 16777216 sc 12 2024-09-17T21:48:24.934 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:24.932+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:24.951 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:24.951 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:24.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:25.035 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:25.033+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:25.107 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:25.107 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:25.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:25.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:25.159 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:25.157+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:25.227 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:25.227 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:25.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:25.237 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:25.235+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:25.284 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:25.284 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:25.301 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:25.301 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:25.318 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:25.318 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:25.318 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:25.335 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:25.335 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:25.337 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:25.335+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:25.346 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:25.347 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:25.347 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:25.347 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:25.347 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:25.352 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:25.352 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:25.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:25.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:25.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:25.439 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:25.437+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:25.505 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.818773381, 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.506 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:25.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:25.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.78649840600001, 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.512 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:25.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:25.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 123.786259638, 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:25.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.618753507999998, 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:25.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:25.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:25.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.611851736999995, 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:25.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:25.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.611735646, 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:25.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:25.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414732125999997, 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:25.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414693331000002, 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:25.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414679047, 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:25.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:25.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:25.559 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414434081000003, 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:25.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.564 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414351633000003, 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:25.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:25.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.569 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414319823, 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:25.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.414296104000002, 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:25.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.579 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.027656706999998, 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:25.580 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:25.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:25.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:25.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.017727727, 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:25.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.592 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825393012999999, 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:25.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:25.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.599 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825135591999999, 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:25.600 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:25.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825102745999999, 2024-09-17T21:48:25.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:25.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:25.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825085801, 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:25.613 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:25.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:25.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 17.825044303999999, 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:25.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:25.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:25.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:25.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:25.626 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:25.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:25.628 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:25.625+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:25.728 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:25.726+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:25.784 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:25.784 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:25.784 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:25.784 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:25.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:25.834 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:25.832+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:25.873 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone47 from 0xcb2b400 (overlap 0x55c3e00) to 0x224e400 2024-09-17T21:48:25.930 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:25.927+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:25.943 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:25.943 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:25.943 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:25.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:26.031 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:26.028+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:26.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.017632170999999, 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:26.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:26.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:26.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:26.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016489882999998, 2024-09-17T21:48:26.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:26.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:26.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016473838, 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:26.114 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:26.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:26.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016457292999998, 2024-09-17T21:48:26.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:26.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:26.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016442527999999, 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:26.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:26.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016415222999999, 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016378334999999, 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:26.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016339637000002, 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:26.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016301464000001, 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:26.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:26.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016264034999999, 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:26.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016207011999999, 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016176342000001, 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:26.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:26.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016031969, 2024-09-17T21:48:26.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:26.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:26.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:26.171 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.016013124000001, 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:26.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.015986527999999, 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:26.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.015951689000001, 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.441448722000001, 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.186 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:26.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:26.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.441415407000001, 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:26.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:26.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:26.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:26.197 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.434325343000001, 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:26.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:26.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:26.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:26.202 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.434291663, 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.207 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:26.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:26.209 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:26.206+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:26.309 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:26.307+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:26.359 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:26.359 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:26.360 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:26.360 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:26.368 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:26.391 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-09-17T21:48:26.410 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:26.407+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:26.511 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:26.508+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:26.537 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:26.537 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:26.537 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:26.537 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:26.537 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:26.546 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:26.580 INFO:teuthology.orchestra.run.smithi165.stdout:5533 write 0x2fc7400 thru 0x2fc76ee (0x2ef bytes) 2024-09-17T21:48:26.581 INFO:teuthology.orchestra.run.smithi165.stdout:5534 write 0x5bc3c00 thru 0x5bce6e9 (0xaaea bytes) 2024-09-17T21:48:26.582 INFO:teuthology.orchestra.run.smithi165.stdout:5535 read 0x1972600 thru 0x197ec60 (0xc661 bytes) 2024-09-17T21:48:26.586 INFO:teuthology.orchestra.run.smithi165.stdout:5536 trunc from 0xcb2b400 to 0xe883e00 2024-09-17T21:48:26.605 INFO:teuthology.orchestra.run.smithi165.stdout:5537 read 0x9b5d400 thru 0x9b6923c (0xbe3d bytes) 2024-09-17T21:48:26.612 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:26.610+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:26.619 INFO:teuthology.orchestra.run.smithi165.stdout:5538 trunc from 0xe883e00 to 0x411b600 2024-09-17T21:48:26.692 INFO:teuthology.orchestra.run.smithi165.stdout:5541 trunc from 0x411b600 to 0x83ae600 2024-09-17T21:48:26.705 INFO:teuthology.orchestra.run.smithi165.stdout:5549 write 0x3d39e00 thru 0x3d449f2 (0xabf3 bytes) 2024-09-17T21:48:26.709 INFO:teuthology.orchestra.run.smithi165.stdout:5550 trunc from 0x83ae600 to 0xa3b6200 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:26.710 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 111.83377332000001, 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:26.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:26.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:26.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:26.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.716 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797327009999997, 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:26.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:26.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797305709, 2024-09-17T21:48:26.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:26.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:26.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797284947999998, 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.730 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797263842, 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.731 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797243352999999, 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797219255999998, 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797196763000002, 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797169156000002, 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 39.797144316000001, 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.645190540999998, 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:26.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.645166779, 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.765 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:26.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.645124389999999, 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:26.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589375736999997, 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589352642000001, 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589326262, 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589309626000002, 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:26.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:26.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589279662000003, 2024-09-17T21:48:26.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:26.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:26.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589266737000003, 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:26.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.589256515999999, 2024-09-17T21:48:26.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:26.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:26.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:26.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:26.805 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:26.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:48:26.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:26.808 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:26.803+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:26.808 INFO:teuthology.orchestra.run.smithi165.stdout:5551 punch from 0x4e40000 to 0x4e46e00, (0x6e00 bytes) 2024-09-17T21:48:26.808 INFO:teuthology.orchestra.run.smithi165.stdout:5556 punch from 0x9a1a00 to 0x9ad200, (0xb800 bytes) 2024-09-17T21:48:26.808 INFO:teuthology.orchestra.run.smithi165.stdout:5560 write 0x5c59800 thru 0x5c63e35 (0xa636 bytes) 2024-09-17T21:48:26.808 INFO:teuthology.orchestra.run.smithi165.stdout:5561 punch from 0x7ac7800 to 0x7ad4200, (0xca00 bytes) 2024-09-17T21:48:26.809 INFO:teuthology.orchestra.run.smithi165.stdout:5563 write 0x6fa5a00 thru 0x6fb09b5 (0xafb6 bytes) 2024-09-17T21:48:26.809 INFO:teuthology.orchestra.run.smithi165.stdout:5565 write 0xe34e600 thru 0xe3557cd (0x71ce bytes) 2024-09-17T21:48:26.809 INFO:teuthology.orchestra.run.smithi165.stdout:5566 trunc from 0xe355800 to 0x1e1a200 2024-09-17T21:48:26.809 INFO:teuthology.orchestra.run.smithi165.stdout:5568 read 0x1223600 thru 0x122fd8b (0xc78c bytes) 2024-09-17T21:48:26.809 INFO:teuthology.orchestra.run.smithi165.stdout:5571 read 0x202200 thru 0x21059d (0xe39e bytes) 2024-09-17T21:48:26.824 INFO:teuthology.orchestra.run.smithi165.stdout:5583 write 0x8ef8000 thru 0x8ef9b08 (0x1b09 bytes) 2024-09-17T21:48:26.828 INFO:teuthology.orchestra.run.smithi165.stdout:5584 clone 49 order 25 su 2097152 sc 13 2024-09-17T21:48:26.906 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:26.904+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:26.916 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:26.916 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:26.916 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:26.916 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:26.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:27.008 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:27.005+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:27.031 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:27.031 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:27.031 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:27.031 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:27.031 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:27.039 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:27.123 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:27.121+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:27.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.446325427, 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:27.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 121.445280967, 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:27.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:27.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.244095635999997, 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:27.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:27.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:27.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:27.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:27.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:27.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.028177968000001, 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:27.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:27.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:27.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.825741226000002, 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.825684045999999, 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:27.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:27.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.825113465000001, 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.825085530999999, 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:27.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:27.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.824951178999999, 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:27.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.824929644000001, 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.239 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:27.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:27.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:27.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.824768679000002, 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:27.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.824712913999999, 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.250 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:27.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:27.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:27.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.823366307000001, 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:27.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:27.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.823314751000002, 2024-09-17T21:48:27.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:27.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:27.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:27.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.822626490000001, 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:27.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:27.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:27.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.822609439000001, 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:27.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.821671199000001, 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:27.276 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:27.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:27.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:27.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:27.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.821640858999999, 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:27.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:27.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.527243411000001, 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:27.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:27.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.626125711, 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:27.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:27.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:27.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:27.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:27.298 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:27.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:27.300 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:27.296+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:27.358 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:27.399 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:27.397+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:27.447 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:27.448 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:27.448 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:27.456 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:27.481 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:27.481 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:27.499 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:27.499 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:27.501 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:27.498+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:27.515 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:27.515 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:27.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:27.549 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:27.549 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:27.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:27.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:27.601 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:27.599+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:27.610 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:27.611 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:27.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:27.703 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:27.700+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:27.730 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:27.730 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:27.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:27.747 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:48:27.805 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:27.803+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:27.856 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:27.856 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:27.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:27.884 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone48 from 0x8ef9c00 (overlap 0x1e1a200) to 0x605200 2024-09-17T21:48:27.907 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:27.905+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.005 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:28.003+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.016 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.016 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.026 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:28.110 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:28.108+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.177 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.177 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:28.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:28.241 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:28.238+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:28.302 INFO:teuthology.orchestra.run.smithi053.stderr:no valid command found; 10 closest matches: 2024-09-17T21:48:28.302 INFO:teuthology.orchestra.run.smithi053.stderr:0 2024-09-17T21:48:28.302 INFO:teuthology.orchestra.run.smithi053.stderr:1 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:2 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:abort 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:assert 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs files list 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:bluefs stats 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:bluestore bluefs device info [] 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:config diff 2024-09-17T21:48:28.303 INFO:teuthology.orchestra.run.smithi053.stderr:admin_socket: invalid command 2024-09-17T21:48:28.304 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:48:28.305 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-3, ['dump_ops_in_flight'] 2024-09-17T21:48:28.308 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:28.306+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:28.336 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.336 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:28.429 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:28.427+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.436 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-09-17T21:48:28.504 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.504 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.509 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:28.506+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:28.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:28.600 INFO:teuthology.orchestra.run.smithi165.stdout:5587 read 0x57f0a00 thru 0x57fa485 (0x9a86 bytes) 2024-09-17T21:48:28.601 INFO:teuthology.orchestra.run.smithi165.stdout:5588 write 0x66eb000 thru 0x66fab0b (0xfb0c bytes) 2024-09-17T21:48:28.601 INFO:teuthology.orchestra.run.smithi165.stdout:5589 write 0x3404a00 thru 0x340bfbd (0x75be bytes) 2024-09-17T21:48:28.601 INFO:teuthology.orchestra.run.smithi165.stdout:5590 read 0x8b59400 thru 0x8b6780f (0xe410 bytes) 2024-09-17T21:48:28.602 INFO:teuthology.orchestra.run.smithi165.stdout:5593 punch from 0x2de0800 to 0x2de7200, (0x6a00 bytes) 2024-09-17T21:48:28.610 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:28.607+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:28.618 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:28.614+0000 7fcf3053e640 -1 osd.3 95 log_to_monitors true 2024-09-17T21:48:28.635 INFO:teuthology.orchestra.run.smithi165.stdout:5595 write 0xc4f5600 thru 0xc4f6111 (0xb12 bytes) 2024-09-17T21:48:28.636 INFO:teuthology.orchestra.run.smithi165.stdout:5596 write 0x3d66e00 thru 0x3d68c69 (0x1e6a bytes) 2024-09-17T21:48:28.636 INFO:teuthology.orchestra.run.smithi165.stdout:5597 write 0x4f11000 thru 0x4f200dd (0xf0de bytes) 2024-09-17T21:48:28.637 INFO:teuthology.orchestra.run.smithi165.stdout:5598 write 0x6523400 thru 0x6531466 (0xe067 bytes) 2024-09-17T21:48:28.637 INFO:teuthology.orchestra.run.smithi165.stdout:5599 clone 50 order 24 su 4194304 sc 9 2024-09-17T21:48:28.665 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.665 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.674 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:28.711 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:28.708+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:28.784 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:28.780+0000 7fcf242dc640 -1 osd.3 147 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T21:48:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.787 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:28.841 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:28.839+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.888 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone49 from 0xc4f6200 (overlap 0x8ef9c00) to 0xe4c200 2024-09-17T21:48:28.909 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:28.909 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:28.913 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:28.911+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:28.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:29.014 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:29.010+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:29.067 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:29.068 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:29.077 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:29.120 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:29.117+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:29.185 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:29.186 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:29.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:29.235 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:29.232+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:29.316 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:29.314+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:29.345 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:29.345 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:29.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:29.417 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:29.414+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:29.463 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:29.463 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:29.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:29.472 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-09-17T21:48:29.518 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:29.516+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:29.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:29.560 INFO:teuthology.orchestra.run.smithi165.stdout:5600 punch from 0x6c9d200 to 0x6ca1e00, (0x4c00 bytes) 2024-09-17T21:48:29.605 INFO:teuthology.orchestra.run.smithi165.stdout:5601 read 0x6e44600 thru 0x6e4c398 (0x7d99 bytes) 2024-09-17T21:48:29.605 INFO:teuthology.orchestra.run.smithi165.stdout:5603 punch from 0x1513000 to 0x151c600, (0x9600 bytes) 2024-09-17T21:48:29.611 INFO:teuthology.orchestra.run.smithi165.stdout:5605 write 0xc00b800 thru 0xc011973 (0x6174 bytes) 2024-09-17T21:48:29.612 INFO:teuthology.orchestra.run.smithi165.stdout:5607 clone 51 order 18 su 8192 sc 12 2024-09-17T21:48:29.620 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:29.617+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:29.630 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:29.630 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:29.639 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:29.677 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:29.678 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:29.694 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:29.694 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:29.710 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:29.711 INFO:teuthology.orchestra.run.smithi007.stderr:Error ENXIO: problem getting command descriptions from osd.3 2024-09-17T21:48:29.723 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:29.720+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:29.727 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:29.727 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:29.744 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:29.744 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:29.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-17T21:48:29.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-17T21:48:29.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:48:29.793 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:29.793 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:29.793 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:29.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:29.823 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:29.820+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:29.924 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:29.920+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:29.950 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97662720899999, 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.951 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:29.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.976604716, 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:29.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:29.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97658045599999, 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:29.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:29.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:29.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97655457299999, 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:29.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:29.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:29.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.976529572, 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.972 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.976489773, 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.973 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97647402800001, 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:29.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.981 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97646249300001, 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:29.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:29.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.976452193, 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:29.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:29.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:29.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:29.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97448094999999, 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.990 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:29.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97446780199999, 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.994 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:29.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:29.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:29.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.974457196, 2024-09-17T21:48:29.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:29.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:30.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:30.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97444685799999, 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:30.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:30.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97443685100001, 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.9744269, 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:30.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:30.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.974411592, 2024-09-17T21:48:30.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.019 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97243570500001, 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:30.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.972424749, 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:30.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.97241226, 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:30.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:30.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.972401797, 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:30.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:30.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:30.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:30.038 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:30.039 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:30.036+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:30.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:30.140 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:30.138+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:30.148 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:30.148 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:30.148 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:30.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:30.241 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:30.239+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:30.264 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:30.264 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:30.272 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:30.357 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:30.355+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:30.421 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:30.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 128.734983199, 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:30.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:30.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:30.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 128.70270822399999, 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:30.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 128.70246945599999, 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:30.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:30.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.534963325999996, 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:30.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:30.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:30.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.443 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.528061554999994, 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 79.527945463999998, 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330941944000003, 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:30.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:30.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:30.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330903149000001, 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:30.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330888864999999, 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:30.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:30.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330643899000002, 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330561451000001, 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:30.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:30.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330529640999998, 2024-09-17T21:48:30.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:30.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:30.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:30.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.330505922, 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:30.482 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.486 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.943866524999997, 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:30.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.933937544999999, 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:30.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:30.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:30.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:30.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.741602831000002, 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:30.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:30.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.741345410000001, 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:30.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:30.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:30.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:30.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.741312564000001, 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:30.510 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:30.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:30.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:30.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.741295618999999, 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:30.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:30.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:30.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:30.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.741254122000001, 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:30.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:30.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:30.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.527 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:30.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:30.529 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:30.526+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:30.629 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:30.626+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91923164000000002, 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91925327599999995, 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:30.676 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:30.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91913862099999999, 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91920909500000003, 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.91650867400000002, 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91662023299999995, 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.57296597100000002, 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.57311751, 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:30.690 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:30.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 4 2024-09-17T21:48:30.694 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:30.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:30.736 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:30.734+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:48:30.829 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:30.827+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:30.849 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:30.849 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:30.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:30.941 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:30.939+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:31.005 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:31.005 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.919288975999997, 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:31.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:31.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:31.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13836 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918146688, 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012279199999997, 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.011 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13836 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084187+0000", 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.15e-07 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084189+0000", 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8109999999999999e-06 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084194+0000", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0939999999999998e-06 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084198+0000", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0559999999999998e-06 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084665+0000", 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00046674599999999998 2024-09-17T21:48:31.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084668+0000", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.399e-06 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074302+0000", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98963406700000001 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074309+0000", 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3039999999999999e-06 2024-09-17T21:48:31.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918130642999998, 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.016 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:31.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:31.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918114097999997, 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:31.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:31.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918099333000001, 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:31.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918072027999997, 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:31.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:31.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:31.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.918035140000001, 2024-09-17T21:48:31.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:31.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917996442000003, 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:31.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:31.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:31.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917958269000003, 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:31.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:31.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13844 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917920840000001, 2024-09-17T21:48:31.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012204000000004, 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13844 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:31.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084412+0000", 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084413+0000", 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9799999999999999e-07 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084415+0000", 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1519999999999999e-06 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084421+0000", 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999997e-06 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084426+0000", 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9899999999999997e-06 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084978+0000", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055182699999999996 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084987+0000", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.405e-06 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074527+0000", 2024-09-17T21:48:31.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98954010400000003 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074534+0000", 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6739999999999997e-06 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13846 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917863816999997, 2024-09-17T21:48:31.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012159700000002, 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13846 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:31.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084469+0000", 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084470+0000", 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.34e-07 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084472+0000", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9520000000000001e-06 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084478+0000", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.784e-06 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084482+0000", 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4859999999999999e-06 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085093+0000", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00061105000000000003 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085101+0000", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0970000000000006e-06 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074584+0000", 2024-09-17T21:48:31.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.989482478 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074591+0000", 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4159999999999998e-06 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13847 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:31.062 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917833147000003, 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011894899999997, 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13847 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084500+0000", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084501+0000", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000001e-07 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084507+0000", 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1920000000000002e-06 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084515+0000", 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2500000000000006e-06 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084521+0000", 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6810000000000003e-06 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085137+0000", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000615972 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085144+0000", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2300000000000002e-06 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074612+0000", 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98946823299999997 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074619+0000", 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9609999999999999e-06 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13852 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917688773999998, 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012226400000003, 2024-09-17T21:48:31.067 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13852 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084645+0000", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3999999999999997e-07 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084647+0000", 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0389999999999999e-06 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084653+0000", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.049e-06 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084658+0000", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8579999999999999e-06 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085383+0000", 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00072562099999999997 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085391+0000", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4399999999999999e-06 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074760+0000", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98936945700000001 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074767+0000", 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.46e-06 2024-09-17T21:48:31.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917669928999999, 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:31.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:31.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917643333000001, 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13855 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.917608494, 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99011932400000002, 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13855 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084728+0000", 2024-09-17T21:48:31.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3549999999999999e-06 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084734+0000", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5749999999999997e-06 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084739+0000", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0189999999999997e-06 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085494+0000", 2024-09-17T21:48:31.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075466499999999996 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085503+0000", 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9190000000000005e-06 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074837+0000", 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98933456900000005 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074844+0000", 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8430000000000004e-06 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.343105526999999, 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.086 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:31.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:31.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:31.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.343072211999999, 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:31.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.335982147999999, 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.097 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:31.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.335948468000002, 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.102 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:31.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.107 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:31.108 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:31.109 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:31.106+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:31.209 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:31.206+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:31.256 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:31.256 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:31.257 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:31.257 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:31.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:31.309 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:31.307+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:31.371 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:31.371 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:31.371 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:31.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:31.424 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:31.422+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:31.488 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 116.612702856, 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:31.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:31.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:31.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576256546000003, 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:31.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576235244999999, 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:31.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576214483999998, 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:31.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576193377999999, 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:31.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:31.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576172888999999, 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:31.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:31.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576148791999998, 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:31.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576126299000002, 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:31.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:31.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:31.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576098692000002, 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:31.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.576073852, 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.531 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.424120076999998, 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:31.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:31.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:31.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.424096315, 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:31.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:31.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.424053925999999, 2024-09-17T21:48:31.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:31.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:31.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368305272999997, 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:31.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:31.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368282178000001, 2024-09-17T21:48:31.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:31.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368255798, 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:31.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:31.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368239162000002, 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:31.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:31.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368209198000002, 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:31.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:31.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368196273000002, 2024-09-17T21:48:31.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:31.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:31.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:31.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.368186051999999, 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:31.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:31.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:31.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:31.580 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:31.582 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:31.582 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:31.579+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:31.683 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:31.681+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:31.702 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:31.702 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:31.702 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:31.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:31.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:31.792 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:31.790+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:31.846 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:31.847 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:31.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:31.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:31.889 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:31.900 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:31.898+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:31.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:31.908 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:31.929 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:31.929 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:31.949 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-17T21:48:31.958 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone50 from 0xc4f6200 (overlap 0xc4f6200) to 0xb143e00 2024-09-17T21:48:31.968 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:31.985 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:31.983+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:31.987 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:31.987 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:31.991 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:31.991 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:32.086 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:32.084+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:32.164 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.164 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.174 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:32.187 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:32.185+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:32.288 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:32.286+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:32.336 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.336 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:32.389 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:32.386+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:32.489 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:32.487+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:32.514 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.515 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:32.590 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:32.587+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:32.633 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.633 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:32.727 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:32.725+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:32.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:48:32.795 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.796 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.796 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:32.793+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:32.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:32.891 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:32.889+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:32.956 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:32.956 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:32.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:33.006 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:33.004+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:33.094 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:33.092+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:33.114 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.114 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:33.195 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:33.193+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:33.231 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.231 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:33.295 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:33.293+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:33.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:33.386 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.386 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:33.396 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:33.394+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:33.419 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.419 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.497 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:33.494+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:33.502 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.502 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:33.598 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:33.595+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:33.618 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.618 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:33.699 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:33.697+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:33.734 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.734 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:33.778 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:48:33.803 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-09-17T21:48:33.837 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:33.835+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:33.868 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:33.869 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:33.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:33.916 INFO:teuthology.orchestra.run.smithi165.stdout:5608 punch from 0x9c6c000 to 0x9c6f400, (0x3400 bytes) 2024-09-17T21:48:33.922 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:33.920+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:33.934 INFO:teuthology.orchestra.run.smithi165.stdout:5610 write 0x2845e00 thru 0x2847600 (0x1801 bytes) 2024-09-17T21:48:33.935 INFO:teuthology.orchestra.run.smithi165.stdout:5612 punch from 0x9da5400 to 0x9db1a00, (0xc600 bytes) 2024-09-17T21:48:33.977 INFO:teuthology.orchestra.run.smithi165.stdout:5614 read 0xab86200 thru 0xab8ee4d (0x8c4e bytes) 2024-09-17T21:48:33.978 INFO:teuthology.orchestra.run.smithi165.stdout:5618 read 0x3d8c600 thru 0x3d99a9e (0xd49f bytes) 2024-09-17T21:48:33.979 INFO:teuthology.orchestra.run.smithi165.stdout:5620 write 0xf3c200 thru 0xf3ec91 (0x2a92 bytes) 2024-09-17T21:48:33.980 INFO:teuthology.orchestra.run.smithi165.stdout:5622 write 0x9bdce00 thru 0x9bddad8 (0xcd9 bytes) 2024-09-17T21:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout:5624 write 0x8b51a00 thru 0x8b59dd0 (0x83d1 bytes) 2024-09-17T21:48:33.982 INFO:teuthology.orchestra.run.smithi165.stdout:5628 read 0x2dfa600 thru 0x2dfbb14 (0x1515 bytes) 2024-09-17T21:48:33.983 INFO:teuthology.orchestra.run.smithi165.stdout:5629 punch from 0x20e3400 to 0x20e7600, (0x4200 bytes) 2024-09-17T21:48:33.999 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:34.002 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:33.999+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:34.007 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:34.007 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:34.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:34.021 INFO:teuthology.orchestra.run.smithi165.stdout:5630 write 0x4d46a00 thru 0x4d46caf (0x2b0 bytes) 2024-09-17T21:48:34.025 INFO:teuthology.orchestra.run.smithi165.stdout:5631 trunc from 0xc4f6200 to 0x99e7200 2024-09-17T21:48:34.093 INFO:teuthology.orchestra.run.smithi165.stdout:5632 write 0x22bf200 thru 0x22c7d09 (0x8b0a bytes) 2024-09-17T21:48:34.102 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:34.100+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:34.137 INFO:teuthology.orchestra.run.smithi165.stdout:5634 write 0xee21400 thru 0xee21923 (0x524 bytes) 2024-09-17T21:48:34.138 INFO:teuthology.orchestra.run.smithi165.stdout:5635 read 0x8572800 thru 0x85796d9 (0x6eda bytes) 2024-09-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi165.stdout:5637 read 0x9b9c800 thru 0x9b9df3c (0x173d bytes) 2024-09-17T21:48:34.139 INFO:teuthology.orchestra.run.smithi165.stdout:5640 punch from 0xb601400 to 0xb60d000, (0xbc00 bytes) 2024-09-17T21:48:34.140 INFO:teuthology.orchestra.run.smithi165.stdout:5643 write 0x5ac2a00 thru 0x5ad2290 (0xf891 bytes) 2024-09-17T21:48:34.153 INFO:teuthology.orchestra.run.smithi165.stdout:5644 trunc from 0xee21a00 to 0x3fa4400 2024-09-17T21:48:34.199 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:34.200 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:34.203 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:34.200+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:34.211 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:34.224 INFO:teuthology.orchestra.run.smithi165.stdout:5646 read 0x3740000 thru 0x374e4bd (0xe4be bytes) 2024-09-17T21:48:34.225 INFO:teuthology.orchestra.run.smithi165.stdout:5648 clone 52 order 23 su 65536 sc 15 2024-09-17T21:48:34.304 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:34.301+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:34.320 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:34.320 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:34.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:34.333 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:34.333 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:34.350 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:34.350 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:34.366 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:34.367 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:34.383 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:34.383 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:34.400 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:34.400 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:34.404 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:34.402+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:34.418 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:34.418 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:34.437 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:34.437 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:34.437 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:34.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:34.505 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:34.503+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.579934331, 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:34.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:34.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57991183799999, 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.559 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:34.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:34.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57988757800001, 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:34.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57986169500001, 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:34.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57983669399999, 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:34.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57979689499999, 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:34.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:34.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57978115, 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:34.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.579769615, 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57975931499999, 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:34.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.593 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.577788072, 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:34.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.597 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57777492400001, 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:34.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:34.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57776431799999, 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:34.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57775398000001, 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.577743973, 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:34.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57773402199999, 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:34.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:34.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:34.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57771871400001, 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:34.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.575742827, 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57573187099999, 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57571938199999, 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 137.57570891899999, 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.643 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:34.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:34.645 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:34.642+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:34.745 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:34.742+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:34.794 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:2548 3.a 3:51392dba:::rbd_object_map.1112986ad02f.0000000000000036:head [writefull 0~100 in=100b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.792017+0000", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00044903800000000002, 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046862899999999997, 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for sub ops", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 2548 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792017+0000", 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792017+0000", 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792018+0000", 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6899999999999996e-07 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792022+0000", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5379999999999996e-06 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792033+0000", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0471e-05 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792039+0000", 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7799999999999997e-06 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792087+0000", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8779000000000002e-05 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792220+0000", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013223400000000001 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.792362+0000", 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000142278 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:48:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:48:34.799 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:34.803 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:34.846 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:34.844+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:34.946 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone51 from 0x3fa4400 (overlap 0x3fa4400) to 0x2c8c800 2024-09-17T21:48:34.946 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:34.944+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:34.964 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:34.965 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:34.965 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:34.965 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:34.965 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:34.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:35.064 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:35.061+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:35.148 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:35.146+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:35.160 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3619 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5529600~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18548518899999999, 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029125908999999998, 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3619 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971880+0000", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8950000000000001e-06 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971886+0000", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8760000000000002e-06 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971890+0000", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6160000000000004e-06 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000819+0000", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028928762 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000896+0000", 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7145999999999996e-05 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001003+0000", 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010726900000000001 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:35.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18546716699999999, 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:48:35.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18544303000000001, 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:48:35.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:48:35.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:35.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.185410775, 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:35.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:48:35.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:35.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.185380722, 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:35.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18535737599999999, 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:48:35.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:48:35.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18530442899999999, 2024-09-17T21:48:35.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:48:35.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.185271083, 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3975 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18196349000000001, 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029182326000000001, 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3975 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975400+0000", 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975402+0000", 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8840000000000001e-06 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975407+0000", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975412+0000", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004419+0000", 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029007848999999999 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004494+0000", 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4351999999999996e-05 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004582+0000", 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7769000000000006e-05 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18193984999999999, 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18191194999999999, 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:35.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:48:35.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18188924400000001, 2024-09-17T21:48:35.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18187297599999999, 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:48:35.215 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:48:35.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18186300499999999, 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:48:35.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18185085600000001, 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:35.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:35.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.181841161, 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:48:35.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:35.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.181831568, 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:48:35.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:48:35.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.181822078, 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:48:35.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18181223199999999, 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.18180289199999999, 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:35.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:35.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:48:35.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:48:35.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:35.250 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:35.251 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:35.249+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:35.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:35.352 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:35.350+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:35.360 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:35.360 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:35.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:35.453 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:35.451+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:35.477 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:35.477 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:35.477 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:35.477 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:35.477 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:35.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:35.554 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:35.551+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:35.597 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:35.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.91129217100001, 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:35.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:35.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:35.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.87901719600001, 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.878778428, 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:35.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 84.711272297999997, 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 84.704370526999995, 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.620 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 84.704254435999999, 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.625 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:35.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:35.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.632 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.507250915999997, 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:35.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.507212121000002, 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:35.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:35.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.507197837, 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.644 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:35.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.648 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.506952871000003, 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.506870423000002, 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:35.653 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.506838612999999, 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:35.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:35.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.506814894000001, 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.663 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:35.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.120175496999998, 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:35.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:35.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:35.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:35.673 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.110246517, 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:35.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:35.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:35.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.917911802999999, 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:35.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:35.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:35.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:35.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:35.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.917654381999998, 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:35.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.917621535999999, 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:35.693 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:35.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:35.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:35.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:35.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:35.700 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.917604591, 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:35.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:35.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:35.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.917563093999998, 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.708 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:35.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:35.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.713 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:35.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:35.714 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:35.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:35.716 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:35.713+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:35.731 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-09-17T21:48:35.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:35.842 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:35.840+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:35.874 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:35.874 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:35.874 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:35.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:35.925 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:35.923+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:36.017 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:36.015+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:36.040 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:36.041 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:36.041 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:36.041 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:36.041 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:36.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:36.135 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:36.133+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:36.206 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.120120647, 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.207 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:36.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.212 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118962314000001, 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:36.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:36.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118945769, 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:36.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118931003999997, 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:36.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118903699000001, 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:36.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118866810999997, 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.233 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:36.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:36.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118828112999999, 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.238 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:36.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118789939999999, 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.243 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:36.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:36.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118501600000002, 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.248 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:36.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:36.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:36.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 38.118475003999997, 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:36.253 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:36.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:36.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.543937198000002, 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:36.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:36.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:36.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:36.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.543903882999999, 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:36.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.536813819000002, 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.270 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:36.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:36.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:36.275 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 32.536780139000001, 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:36.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:36.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.4483112089999999, 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:36.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:48:36.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.4482181900000004, 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:36.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:36.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:48:36.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.4455882429999996, 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:36.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:36.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:36.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:48:36.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:48:36.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 6.1020455399999998, 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:36.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.446726043, 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:36.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:48:36.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 5.4408170130000002, 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:36.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:48:36.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:48:36.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:36.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:36.321 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:36.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:36.322 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:36.320+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:36.378 INFO:teuthology.orchestra.run.smithi165.stdout:5649 trunc from 0x3fa4400 to 0x6e59200 2024-09-17T21:48:36.392 INFO:teuthology.orchestra.run.smithi165.stdout:5650 write 0xea0800 thru 0xeaf9be (0xf1bf bytes) 2024-09-17T21:48:36.397 INFO:teuthology.orchestra.run.smithi165.stdout:5651 write 0x15a4e00 thru 0x15b18a0 (0xcaa1 bytes) 2024-09-17T21:48:36.398 INFO:teuthology.orchestra.run.smithi165.stdout:5652 write 0x6b9e400 thru 0x6ba73fc (0x8ffd bytes) 2024-09-17T21:48:36.398 INFO:teuthology.orchestra.run.smithi165.stdout:5654 read 0x441bc00 thru 0x442586f (0x9c70 bytes) 2024-09-17T21:48:36.402 INFO:teuthology.orchestra.run.smithi165.stdout:5656 trunc from 0x6e59200 to 0x30e4200 2024-09-17T21:48:36.423 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:36.420+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:36.430 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:36.493 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:36.493 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:36.493 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:36.493 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:36.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:36.547 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:36.544+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:36.591 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:36.591 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:36.613 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:36.614 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:36.624 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:36.622+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:36.630 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:36.630 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:36.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:36.650 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:36.674 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:36.674 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:36.698 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:36.699 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:36.755 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:36.753+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:36.783 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:36.784 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:36.788 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:48:36.794 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:36.826 INFO:teuthology.orchestra.run.smithi165.stdout:5657 write 0x8ca7000 thru 0x8ca7a6a (0xa6b bytes) 2024-09-17T21:48:36.832 INFO:teuthology.orchestra.run.smithi165.stdout:5659 read 0x7923800 thru 0x792e2de (0xaadf bytes) 2024-09-17T21:48:36.832 INFO:teuthology.orchestra.run.smithi165.stdout:5660 read 0x5c8600 thru 0x5cacf4 (0x26f5 bytes) 2024-09-17T21:48:36.833 INFO:teuthology.orchestra.run.smithi165.stdout:5661 write 0x5d2a400 thru 0x5d2b0e9 (0xcea bytes) 2024-09-17T21:48:36.833 INFO:teuthology.orchestra.run.smithi165.stdout:5663 read 0x381d600 thru 0x38264aa (0x8eab bytes) 2024-09-17T21:48:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:5665 punch from 0x2df1800 to 0x2dfbe00, (0xa600 bytes) 2024-09-17T21:48:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:5666 read 0x1f35c00 thru 0x1f3fa86 (0x9e87 bytes) 2024-09-17T21:48:36.839 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:36.837+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:36.850 INFO:teuthology.orchestra.run.smithi165.stdout:5667 write 0xdd72e00 thru 0xdd7e613 (0xb814 bytes) 2024-09-17T21:48:36.858 INFO:teuthology.orchestra.run.smithi165.stdout:5669 read 0x3888e00 thru 0x389504b (0xc24c bytes) 2024-09-17T21:48:36.858 INFO:teuthology.orchestra.run.smithi165.stdout:truncating to largest ever: 0xedfd200 2024-09-17T21:48:36.859 INFO:teuthology.orchestra.run.smithi165.stdout:5670 trunc from 0xdd7e800 to 0xedfd200 2024-09-17T21:48:36.873 INFO:teuthology.orchestra.run.smithi165.stdout:5673 write 0xb89c200 thru 0xb8a7a00 (0xb801 bytes) 2024-09-17T21:48:36.874 INFO:teuthology.orchestra.run.smithi165.stdout:5674 read 0xaa33a00 thru 0xaa33fe6 (0x5e7 bytes) 2024-09-17T21:48:36.874 INFO:teuthology.orchestra.run.smithi165.stdout:5675 write 0x658d000 thru 0x65930ee (0x60ef bytes) 2024-09-17T21:48:36.874 INFO:teuthology.orchestra.run.smithi165.stdout:5680 punch from 0x255ac00 to 0x2567a00, (0xce00 bytes) 2024-09-17T21:48:36.875 INFO:teuthology.orchestra.run.smithi165.stdout:5682 read 0xafca800 thru 0xafd839b (0xdb9c bytes) 2024-09-17T21:48:36.875 INFO:teuthology.orchestra.run.smithi165.stdout:5683 punch from 0x5921a00 to 0x5930800, (0xee00 bytes) 2024-09-17T21:48:36.875 INFO:teuthology.orchestra.run.smithi165.stdout:5684 write 0x59b1c00 thru 0x59b7e84 (0x6285 bytes) 2024-09-17T21:48:36.877 INFO:teuthology.orchestra.run.smithi165.stdout:5685 read 0x5ac9800 thru 0x5aca575 (0xd76 bytes) 2024-09-17T21:48:36.877 INFO:teuthology.orchestra.run.smithi165.stdout:5688 write 0xd94f400 thru 0xd954c86 (0x5887 bytes) 2024-09-17T21:48:36.878 INFO:teuthology.orchestra.run.smithi165.stdout:5690 punch from 0xa4ade00 to 0xa4b2400, (0x4600 bytes) 2024-09-17T21:48:36.878 INFO:teuthology.orchestra.run.smithi165.stdout:5691 write 0x1b28200 thru 0x1b363cc (0xe1cd bytes) 2024-09-17T21:48:36.883 INFO:teuthology.orchestra.run.smithi165.stdout:5693 trunc from 0xedfd200 to 0xbd13a00 2024-09-17T21:48:36.898 INFO:teuthology.orchestra.run.smithi165.stdout:5694 read 0x5494600 thru 0x5497486 (0x2e87 bytes) 2024-09-17T21:48:36.899 INFO:teuthology.orchestra.run.smithi165.stdout:5696 read 0x9c81200 thru 0x9c887ef (0x75f0 bytes) 2024-09-17T21:48:36.900 INFO:teuthology.orchestra.run.smithi165.stdout:5697 punch from 0x260fa00 to 0x261e800, (0xee00 bytes) 2024-09-17T21:48:36.900 INFO:teuthology.orchestra.run.smithi165.stdout:5699 read 0x6469800 thru 0x6478004 (0xe805 bytes) 2024-09-17T21:48:36.902 INFO:teuthology.orchestra.run.smithi165.stdout:5702 trunc from 0xbd13a00 to 0xcecc400 2024-09-17T21:48:36.911 INFO:teuthology.orchestra.run.smithi165.stdout:5708 write 0x8280800 thru 0x829004f (0xf850 bytes) 2024-09-17T21:48:36.918 INFO:teuthology.orchestra.run.smithi165.stdout:5711 read 0x23f3600 thru 0x23f5679 (0x207a bytes) 2024-09-17T21:48:36.921 INFO:teuthology.orchestra.run.smithi165.stdout:5713 trunc from 0xcecc400 to 0xae14400 2024-09-17T21:48:36.926 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:36.926 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:36.927 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:36.924+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:36.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:36.937 INFO:teuthology.orchestra.run.smithi165.stdout:5714 write 0x822ec00 thru 0x823554b (0x694c bytes) 2024-09-17T21:48:36.938 INFO:teuthology.orchestra.run.smithi165.stdout:5717 write 0x2b1ae00 thru 0x2b1bbb4 (0xdb5 bytes) 2024-09-17T21:48:36.939 INFO:teuthology.orchestra.run.smithi165.stdout:5719 read 0x14b4e00 thru 0x14c0ae7 (0xbce8 bytes) 2024-09-17T21:48:36.940 INFO:teuthology.orchestra.run.smithi165.stdout:5720 punch from 0x77c3800 to 0x77d2200, (0xea00 bytes) 2024-09-17T21:48:36.940 INFO:teuthology.orchestra.run.smithi165.stdout:5721 punch from 0x46e6400 to 0x46e7800, (0x1400 bytes) 2024-09-17T21:48:36.940 INFO:teuthology.orchestra.run.smithi165.stdout:5724 write 0x30ed000 thru 0x30f37c1 (0x67c2 bytes) 2024-09-17T21:48:36.942 INFO:teuthology.orchestra.run.smithi165.stdout:5727 trunc from 0xae14400 to 0x9a33000 2024-09-17T21:48:36.956 INFO:teuthology.orchestra.run.smithi165.stdout:5728 read 0x4f73e00 thru 0x4f7ac06 (0x6e07 bytes) 2024-09-17T21:48:36.962 INFO:teuthology.orchestra.run.smithi165.stdout:5729 trunc from 0x9a33000 to 0x6a87e00 2024-09-17T21:48:36.982 INFO:teuthology.orchestra.run.smithi165.stdout:5730 read 0x2489600 thru 0x2496e73 (0xd874 bytes) 2024-09-17T21:48:36.990 INFO:teuthology.orchestra.run.smithi165.stdout:5734 read 0x1c04400 thru 0x1c04cf2 (0x8f3 bytes) 2024-09-17T21:48:36.990 INFO:teuthology.orchestra.run.smithi165.stdout:5736 read 0x36cba00 thru 0x36ce2a5 (0x28a6 bytes) 2024-09-17T21:48:36.990 INFO:teuthology.orchestra.run.smithi165.stdout:5737 read 0x44c400 thru 0x45b399 (0xef9a bytes) 2024-09-17T21:48:36.991 INFO:teuthology.orchestra.run.smithi165.stdout:5739 write 0x2f97800 thru 0x2fa249d (0xac9e bytes) 2024-09-17T21:48:36.991 INFO:teuthology.orchestra.run.smithi165.stdout:5740 read 0x38400 thru 0x3a3bc (0x1fbd bytes) 2024-09-17T21:48:36.992 INFO:teuthology.orchestra.run.smithi165.stdout:5741 read 0x17ade00 thru 0x17bb230 (0xd431 bytes) 2024-09-17T21:48:36.993 INFO:teuthology.orchestra.run.smithi165.stdout:5742 read 0x3f40a00 thru 0x3f486f1 (0x7cf2 bytes) 2024-09-17T21:48:36.993 INFO:teuthology.orchestra.run.smithi165.stdout:5745 read 0x9f5c00 thru 0x9f86fa (0x2afb bytes) 2024-09-17T21:48:37.027 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:37.025+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:37.084 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.085 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:37.128 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:37.126+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:37.199 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.199 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.208 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:37.215 INFO:teuthology.orchestra.run.smithi165.stdout:5747 write 0xa47a600 thru 0xa481c3b (0x763c bytes) 2024-09-17T21:48:37.215 INFO:teuthology.orchestra.run.smithi165.stdout:5748 write 0xe51600 thru 0xe5e5ed (0xcfee bytes) 2024-09-17T21:48:37.217 INFO:teuthology.orchestra.run.smithi165.stdout:5751 write 0x99e9800 thru 0x99f6fe6 (0xd7e7 bytes) 2024-09-17T21:48:37.217 INFO:teuthology.orchestra.run.smithi165.stdout:5752 write 0x720f800 thru 0x7216283 (0x6a84 bytes) 2024-09-17T21:48:37.217 INFO:teuthology.orchestra.run.smithi165.stdout:5753 read 0x5122200 thru 0x512ef2c (0xcd2d bytes) 2024-09-17T21:48:37.217 INFO:teuthology.orchestra.run.smithi165.stdout:5755 write 0x893f800 thru 0x894c14b (0xc94c bytes) 2024-09-17T21:48:37.229 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:37.227+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:37.240 INFO:teuthology.orchestra.run.smithi165.stdout:5757 write 0xcd2c800 thru 0xcd36013 (0x9814 bytes) 2024-09-17T21:48:37.240 INFO:teuthology.orchestra.run.smithi165.stdout:5759 punch from 0x607ce00 to 0x6085000, (0x8200 bytes) 2024-09-17T21:48:37.241 INFO:teuthology.orchestra.run.smithi165.stdout:5760 write 0x5c97a00 thru 0x5ca5009 (0xd60a bytes) 2024-09-17T21:48:37.245 INFO:teuthology.orchestra.run.smithi165.stdout:5761 write 0x3daa800 thru 0x3db217d (0x797e bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5763 punch from 0xba76200 to 0xba84400, (0xe200 bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5765 read 0xbb36200 thru 0xbb3a5f1 (0x43f2 bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5766 punch from 0x1b82a00 to 0x1b8f800, (0xce00 bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5767 read 0x8bf5c00 thru 0x8bf649a (0x89b bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5768 read 0x7b50600 thru 0x7b58d4a (0x874b bytes) 2024-09-17T21:48:37.246 INFO:teuthology.orchestra.run.smithi165.stdout:5770 write 0xbc63e00 thru 0xbc72712 (0xe913 bytes) 2024-09-17T21:48:37.251 INFO:teuthology.orchestra.run.smithi165.stdout:5771 trunc from 0xcd36200 to 0x8177a00 2024-09-17T21:48:37.268 INFO:teuthology.orchestra.run.smithi165.stdout:5773 read 0x4c7f600 thru 0x4c82984 (0x3385 bytes) 2024-09-17T21:48:37.273 INFO:teuthology.orchestra.run.smithi165.stdout:5774 read 0x319c400 thru 0x31a19f4 (0x55f5 bytes) 2024-09-17T21:48:37.274 INFO:teuthology.orchestra.run.smithi165.stdout:5781 punch from 0x2249c00 to 0x224de00, (0x4200 bytes) 2024-09-17T21:48:37.274 INFO:teuthology.orchestra.run.smithi165.stdout:5783 write 0x70b0e00 thru 0x70bb442 (0xa643 bytes) 2024-09-17T21:48:37.274 INFO:teuthology.orchestra.run.smithi165.stdout:5785 read 0x20ce400 thru 0x20d842a (0xa02b bytes) 2024-09-17T21:48:37.293 INFO:teuthology.orchestra.run.smithi165.stdout:5787 write 0xc686c00 thru 0xc68ea8e (0x7e8f bytes) 2024-09-17T21:48:37.293 INFO:teuthology.orchestra.run.smithi165.stdout:5788 read 0x6819600 thru 0x681adc4 (0x17c5 bytes) 2024-09-17T21:48:37.294 INFO:teuthology.orchestra.run.smithi165.stdout:5790 read 0x2349a00 thru 0x234cb89 (0x318a bytes) 2024-09-17T21:48:37.311 INFO:teuthology.orchestra.run.smithi165.stdout:5792 trunc from 0xc68ec00 to 0xef0e00 2024-09-17T21:48:37.330 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:37.327+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:37.342 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.342 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.343 INFO:teuthology.orchestra.run.smithi165.stdout:5793 read 0xaa7800 thru 0xaac5ed (0x4dee bytes) 2024-09-17T21:48:37.352 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:37.367 INFO:teuthology.orchestra.run.smithi165.stdout:5795 write 0x9d29000 thru 0x9d2c38b (0x338c bytes) 2024-09-17T21:48:37.368 INFO:teuthology.orchestra.run.smithi165.stdout:5796 read 0x3415800 thru 0x3417d41 (0x2542 bytes) 2024-09-17T21:48:37.368 INFO:teuthology.orchestra.run.smithi165.stdout:5798 read 0x4d56a00 thru 0x4d5ad33 (0x4334 bytes) 2024-09-17T21:48:37.372 INFO:teuthology.orchestra.run.smithi165.stdout:5801 clone 53 order 23 su 32768 sc 11 2024-09-17T21:48:37.431 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:37.429+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:37.497 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.498 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:37.549 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:37.547+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:37.613 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.613 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.622 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:37.633 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:37.631+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:37.734 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:37.731+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:37.769 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.769 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:37.834 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:37.832+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:37.929 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:37.930 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:37.935 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:37.932+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:37.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:37.966 INFO:teuthology.orchestra.run.smithi165.stdout:leaving image image_client.0-clone52 intact 2024-09-17T21:48:38.035 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:38.033+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:38.089 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.089 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:38.138 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:38.136+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:38.218 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.218 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.227 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:38.237 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:38.234+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:38.338 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.338 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.338 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:38.336+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:38.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:38.428 INFO:tasks.thrashosds.thrasher:in_osds: [1, 3, 5, 4, 0] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:48:38.429 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:38.429 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [1, 3, 5, 4, 0] 2024-09-17T21:48:38.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-17T21:48:38.439 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:38.437+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:38.460 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.460 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:38.540 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:38.537+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:38.621 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.622 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:38.674 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:38.672+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:38.688 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-09-17T21:48:38.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:38.742 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:38.739+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:38.742 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:38.747 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:48:38.745+0000 7f0c28614640 -1 mon.a@0(leader).osd e155 definitely_dead 0 2024-09-17T21:48:38.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.3. 2024-09-17T21:48:38.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:48:38.843 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:38.840+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:38.849 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:38.850 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:38.867 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:38.867 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:38.885 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:38.885 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:38.902 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:38.902 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.825454014999998, 2024-09-17T21:48:38.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:38.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824295681999999, 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:38.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:38.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824279136999998, 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.924 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:38.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:38.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824264372000002, 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.929 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:38.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:38.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824237066999999, 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.934 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:38.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824200179000002, 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:38.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824161480999997, 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:38.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.824123307999997, 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.949 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:38.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:38.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.823834968, 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:38.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 40.823808372000002, 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:38.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:38.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.249270566, 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:38.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:38.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:38.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.249237250999997, 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:38.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:38.976 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.242147187, 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:38.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:38.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.242113506999999, 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1536445769999997, 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:48:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1535515580000002, 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.001 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 9.1509216109999993, 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:39.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:48:39.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:39.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.8073789080000005, 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:39.010 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:48:39.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.1520594109999998, 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:48:39.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:48:39.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 8.146150381, 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.023 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:39.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:48:39.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.029 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:39.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:39.030 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:39.031 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:39.031 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:39.032 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:39.032 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:39.028+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:39.131 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:39.128+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:39.179 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:39.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:39.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:39.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:39.231 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:39.229+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:39.294 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:39.294 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:39.294 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:39.303 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:39.332 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:39.330+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.535806159, 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:39.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:39.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:39.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499359849000001, 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:39.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:39.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499338547999997, 2024-09-17T21:48:39.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:39.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.429 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499317787000003, 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:39.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499296680999997, 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:39.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:39.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499276192000004, 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:39.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499252095000003, 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:39.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499229602, 2024-09-17T21:48:39.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:39.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499201995, 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:39.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.499177154999998, 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:39.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.347223380000003, 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:39.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.347199617999998, 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:39.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.347157228999997, 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291408576000002, 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:39.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:39.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291385480999999, 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:39.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:39.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291359100999998, 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:39.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291342465, 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:39.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291312501, 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:39.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291299576, 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:39.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 48.291289355000004, 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:39.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:39.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:39.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:39.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:39.509 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:39.510 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:39.511 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:39.508+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:39.611 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:39.608+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:39.624 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:39.624 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:39.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:39.712 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:39.710+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:39.740 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:39.740 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:39.740 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:39.740 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:39.740 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:39.748 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:39.799 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:48:39.813 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:39.811+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.862 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.11980220999999, 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.863 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:39.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:39.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 134.11875774999999, 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:39.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:39.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:39.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.917572418999995, 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:39.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:39.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:39.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.701654751, 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:39.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:39.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.499218009000003, 2024-09-17T21:48:39.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:39.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:39.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:39.892 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.499160828999997, 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:39.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498590247999999, 2024-09-17T21:48:39.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:39.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:39.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498562313999997, 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:39.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:39.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:39.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498427962000001, 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:39.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:39.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498406426999999, 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:39.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498245462, 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:39.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:39.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:39.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.498189697000001, 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:39.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:39.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:39.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.932 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.496843089999999, 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:39.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:39.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:39.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.496791534000003, 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:39.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:39.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.496103273000003, 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:39.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.496086222000002, 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:39.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.495147981999999, 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:39.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.495117641999997, 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:39.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.965 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.200720193999999, 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:39.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:39.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.971 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.299602493999998, 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:39.972 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:39.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:39.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:39.977 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:39.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:39.979 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:39.976+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:40.079 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:40.077+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:40.126 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:40.127 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:40.180 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:40.178+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:40.242 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.243 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:40.243 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:40.243 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:40.243 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:40.282 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:40.279+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38568331900001, 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:40.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:40.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38566082599999, 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:40.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:40.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38563656599999, 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:40.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38561068300001, 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.385585682, 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38554588299999, 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38553013800001, 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38551860300001, 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38550830299999, 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38353706000001, 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38352391199999, 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.383513306, 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38350296799999, 2024-09-17T21:48:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.383492961, 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38348300999999, 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38346770199999, 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.381491815, 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38148085899999, 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.38146836999999, 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 143.381457907, 2024-09-17T21:48:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.457 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:40.459 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:40.456+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:40.559 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:40.557+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:40.566 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:40.566 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:40.566 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:40.660 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:40.658+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:40.681 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:40.682 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:40.774 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:40.772+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:40.858 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3619 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5529600~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8834337730000001, 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029125908999999998, 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3619 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971880+0000", 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8950000000000001e-06 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971886+0000", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8760000000000002e-06 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971890+0000", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6160000000000004e-06 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000819+0000", 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028928762 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000896+0000", 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7145999999999996e-05 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001003+0000", 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010726900000000001 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8834157510000002, 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:48:40.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:48:40.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8833916139999998, 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.883359359, 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:48:40.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8833293060000003, 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:48:40.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8833059599999995, 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:48:40.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:48:40.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.883253013, 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8832196669999997, 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:48:40.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:48:40.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3975 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8799120739999999, 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029182326000000001, 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.895 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3975 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975400+0000", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975402+0000", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8840000000000001e-06 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975407+0000", 2024-09-17T21:48:40.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975412+0000", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004419+0000", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029007848999999999 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004494+0000", 2024-09-17T21:48:40.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4351999999999996e-05 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004582+0000", 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7769000000000006e-05 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8798884339999997, 2024-09-17T21:48:40.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:40.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8798605339999996, 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:48:40.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:48:40.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8798378280000003, 2024-09-17T21:48:40.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:48:40.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8798215599999999, 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:48:40.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:48:40.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.879811589, 2024-09-17T21:48:40.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:48:40.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8797994400000002, 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:48:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:48:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8797897450000001, 2024-09-17T21:48:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:48:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:48:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8797801520000004, 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:48:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8797706620000003, 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:48:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:48:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8797608160000001, 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.879751476, 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:40.949 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:40.951 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:40.951 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:40.948+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:41.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:41.050 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:41.048+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3095 3.12 3.1bab772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.099964+0000", 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0084648229999999998, 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0084789689999999994, 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.112 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3095 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099964+0000", 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099964+0000", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099965+0000", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4099999999999998e-07 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099969+0000", 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9319999999999998e-06 2024-09-17T21:48:41.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099978+0000", 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1370000000000008e-06 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.099984+0000", 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4060000000000002e-06 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3097 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100585+0000", 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0078437999999999997, 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078938389999999997, 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3097 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100585+0000", 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100585+0000", 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100586+0000", 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3500000000000005e-07 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100590+0000", 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0450000000000002e-06 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100603+0000", 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2830000000000001e-05 2024-09-17T21:48:41.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100622+0000", 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8839999999999999e-05 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3098 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100635+0000", 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0077937149999999997, 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.007872146, 2024-09-17T21:48:41.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3098 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100635+0000", 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100635+0000", 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100636+0000", 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7299999999999996e-07 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100639+0000", 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0469999999999998e-06 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100647+0000", 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6329999999999996e-06 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100653+0000", 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0100000000000001e-06 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3099 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100661+0000", 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0077682799999999998, 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078738379999999993, 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3099 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100661+0000", 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100661+0000", 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100661+0000", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3099999999999997e-07 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100664+0000", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.238e-06 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100672+0000", 2024-09-17T21:48:41.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3010000000000007e-06 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100678+0000", 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5990000000000001e-06 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3100 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100688+0000", 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0077407040000000002, 2024-09-17T21:48:41.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078731519999999996, 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3100 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100688+0000", 2024-09-17T21:48:41.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100688+0000", 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100689+0000", 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100691+0000", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2239999999999998e-06 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100707+0000", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5979999999999999e-05 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100713+0000", 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9379999999999997e-06 2024-09-17T21:48:41.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3101 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100724+0000", 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0077051239999999998, 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078665790000000003, 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3101 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100724+0000", 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100724+0000", 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100724+0000", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6800000000000001e-07 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100727+0000", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8509999999999999e-06 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100736+0000", 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7490000000000004e-06 2024-09-17T21:48:41.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100742+0000", 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8100000000000003e-06 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3102 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100749+0000", 2024-09-17T21:48:41.133 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076798359999999998, 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078679849999999992, 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3102 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100749+0000", 2024-09-17T21:48:41.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100749+0000", 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100750+0000", 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100752+0000", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100759+0000", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5300000000000002e-06 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100764+0000", 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2700000000000004e-06 2024-09-17T21:48:41.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3103 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100771+0000", 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076584929999999997, 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0078834690000000006, 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3103 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100771+0000", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100771+0000", 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100771+0000", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5000000000000003e-07 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100773+0000", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.221e-06 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100780+0000", 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3280000000000002e-06 2024-09-17T21:48:41.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100785+0000", 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3104 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100793+0000", 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076360739999999996, 2024-09-17T21:48:41.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079258750000000006, 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3104 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100793+0000", 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100793+0000", 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100793+0000", 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0600000000000001e-07 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100796+0000", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100803+0000", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6510000000000001e-06 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100810+0000", 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5370000000000003e-06 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3105 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100818+0000", 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0076109130000000004, 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0079947690000000005, 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3105 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100818+0000", 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100818+0000", 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100819+0000", 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9200000000000002e-07 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100821+0000", 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6060000000000001e-06 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100829+0000", 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5490000000000001e-06 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100834+0000", 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5790000000000003e-06 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3106 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100843+0000", 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0075857549999999996, 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081735539999999995, 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3106 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100843+0000", 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100843+0000", 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100844+0000", 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3300000000000003e-07 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100846+0000", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100858+0000", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.198e-05 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100864+0000", 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7250000000000002e-06 2024-09-17T21:48:41.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3107 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.100971+0000", 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0074580100000000002, 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081048809999999995, 2024-09-17T21:48:41.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3107 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100971+0000", 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100971+0000", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100972+0000", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8599999999999998e-07 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100982+0000", 2024-09-17T21:48:41.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0518e-05 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.100993+0000", 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.061e-05 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101001+0000", 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.9859999999999994e-06 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3108 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.101019+0000", 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0074098250000000001, 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081083769999999999, 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3108 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101019+0000", 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101019+0000", 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101020+0000", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9999999999999998e-07 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101028+0000", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3769999999999996e-06 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101036+0000", 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6190000000000002e-06 2024-09-17T21:48:41.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101042+0000", 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1829999999999999e-06 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3109 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.101052+0000", 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0073768469999999997, 2024-09-17T21:48:41.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081265279999999992, 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3109 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101052+0000", 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101052+0000", 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101053+0000", 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101056+0000", 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0010000000000002e-06 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101074+0000", 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8547999999999999e-05 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101080+0000", 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9909999999999999e-06 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3110 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.101096+0000", 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0073329299999999997, 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081370079999999994, 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3110 2024-09-17T21:48:41.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101096+0000", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101096+0000", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101097+0000", 2024-09-17T21:48:41.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3200000000000005e-07 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101100+0000", 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.061e-06 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101108+0000", 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7609999999999996e-06 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101114+0000", 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6999999999999996e-06 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3111 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.101123+0000", 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.0073065029999999998, 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0081660080000000006, 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3111 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101123+0000", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101123+0000", 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101123+0000", 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9700000000000002e-07 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101126+0000", 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.627e-06 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101132+0000", 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9419999999999998e-06 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.101138+0000", 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3290000000000001e-06 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3122 3.12 3.1bab772 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.168 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.108799+0000", 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.99963, 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055134499999999996, 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3122 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.108799+0000", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.108799+0000", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.108800+0000", 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2399999999999998e-07 2024-09-17T21:48:41.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.108802+0000", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8279999999999999e-06 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.108994+0000", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00019134499999999999 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109001+0000", 2024-09-17T21:48:41.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3690000000000001e-06 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3124 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.109020+0000", 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9994092, 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00039412099999999999, 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3124 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109020+0000", 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109020+0000", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109021+0000", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3399999999999998e-07 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109025+0000", 2024-09-17T21:48:41.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.549e-06 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109036+0000", 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1404e-05 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109043+0000", 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1360000000000002e-06 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5921.0:3125 3.12 3.90b96032 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e158)", 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:41.109066+0000", 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4294967295.9993634, 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00045188699999999999, 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "queued for pg", 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5921", 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1322464496", 2024-09-17T21:48:41.176 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 3125 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109066+0000", 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109066+0000", 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109066+0000", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-07 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109070+0000", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4290000000000001e-06 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109078+0000", 2024-09-17T21:48:41.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0509999999999997e-06 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:41.109084+0000", 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.596e-06 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 19 2024-09-17T21:48:41.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:41.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:41.181 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:41.178+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:41.222 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:41.249 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:41.249 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:41.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:41.276 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:41.281 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:41.278+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:41.305 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:41.306 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:41.336 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:41.337 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:41.362 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:41.382 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:41.382 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:41.380+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:41.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:41.482 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:41.479+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:41.574 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:41.574 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:41.583 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:41.580+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:41.586 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:41.683 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:41.681+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:41.713 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:41.713 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:41.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:41.807 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:41.805+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:41.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:48:41.885 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:41.882+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:41.894 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:41.904 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:41.986 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:41.984+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:42.025 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.025 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:42.087 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:42.084+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:42.182 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.182 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.188 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:42.185+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:42.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:42.288 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:42.286+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:42.297 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.297 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:42.389 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:42.387+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:42.453 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.453 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:42.502 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:42.500+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:42.577 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.577 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:42.590 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:42.588+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:42.691 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:42.688+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:42.692 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.692 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.701 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:42.791 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:42.789+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:42.817 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:48:42.877 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.878 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:42.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:42.891 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:42.889+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:42.993 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:42.990+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:42.997 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:42.997 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:43.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:43.094 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:43.091+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:43.153 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:43.153 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:43.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:43.203 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:43.200+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:43.295 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:43.293+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:43.309 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:43.309 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:43.318 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:43.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:43.422 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:43.420+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:43.465 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:43.465 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:43.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:43.514 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:43.515 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:43.517 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:43.515+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:43.532 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:43.532 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:43.548 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:43.548 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:43.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:43.565 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:43.580 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:43.580 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:43.580 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:43.580 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:43.581 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:43.582 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:43.582 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:43.589 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:43.599 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:43.631 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:43.628+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:43.699 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:43.696+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:43.737 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:43.737 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3619 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5529600~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7624051759999997, 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029125908999999998, 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3619 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971880+0000", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8950000000000001e-06 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971886+0000", 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8760000000000002e-06 2024-09-17T21:48:43.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971890+0000", 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6160000000000004e-06 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000819+0000", 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028928762 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000896+0000", 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7145999999999996e-05 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001003+0000", 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010726900000000001 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7623871540000007, 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:48:43.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:48:43.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:48:43.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7623630170000002, 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7623307619999995, 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.751 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7623007089999998, 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.759 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7622773630000008, 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:43.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:48:43.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7622244160000005, 2024-09-17T21:48:43.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:48:43.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7621910700000001, 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.769 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:48:43.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3975 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7588834769999995, 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029182326000000001, 2024-09-17T21:48:43.773 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3975 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975400+0000", 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975402+0000", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8840000000000001e-06 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975407+0000", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975412+0000", 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:48:43.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004419+0000", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029007848999999999 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004494+0000", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4351999999999996e-05 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004582+0000", 2024-09-17T21:48:43.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7769000000000006e-05 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7588598369999993, 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7588319370000001, 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:43.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7588092310000008, 2024-09-17T21:48:43.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587929629999994, 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:48:43.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:43.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:48:43.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587829920000004, 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:48:43.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587708430000006, 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:43.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:48:43.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587611479999996, 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:48:43.806 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:48:43.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587515549999999, 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.811 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:48:43.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:43.815 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587420649999999, 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:43.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:48:43.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:48:43.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587322190000005, 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:48:43.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7587228790000005, 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:43.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:43.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:43.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:43.829 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:43.830 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:48:43.830 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:43.830 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-17T21:48:43.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-17T21:48:43.832 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:43.828+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:43.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:43.931 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:43.929+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:43.959 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:43.959 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:43.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:44.013 INFO:teuthology.orchestra.run.smithi165.stdout:5802 punch from 0x797f800 to 0x798d400, (0xdc00 bytes) 2024-09-17T21:48:44.031 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:44.029+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:44.049 INFO:teuthology.orchestra.run.smithi165.stdout:5803 write 0xd9c2600 thru 0xd9cef4c (0xc94d bytes) 2024-09-17T21:48:44.050 INFO:teuthology.orchestra.run.smithi165.stdout:5804 read 0xcb8200 thru 0xcbeb8c (0x698d bytes) 2024-09-17T21:48:44.054 INFO:teuthology.orchestra.run.smithi165.stdout:5806 write 0xcbbd600 thru 0xcbc21a5 (0x4ba6 bytes) 2024-09-17T21:48:44.054 INFO:teuthology.orchestra.run.smithi165.stdout:5807 punch from 0x141a200 to 0x1425200, (0xb000 bytes) 2024-09-17T21:48:44.073 INFO:teuthology.orchestra.run.smithi165.stdout:5808 trunc from 0xd9cf000 to 0x1564000 2024-09-17T21:48:44.132 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:44.129+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:44.146 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:44.146 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:44.146 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:44.146 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:44.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:44.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:44.207 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:48:44.204+0000 7f0c28614640 -1 mon.a@0(leader).osd e161 definitely_dead 0 2024-09-17T21:48:44.207 INFO:teuthology.orchestra.run.smithi165.stdout:5810 write 0x79b6e00 thru 0x79c55bd (0xe7be bytes) 2024-09-17T21:48:44.208 INFO:teuthology.orchestra.run.smithi165.stdout:5811 read 0x628e200 thru 0x629a4ef (0xc2f0 bytes) 2024-09-17T21:48:44.212 INFO:teuthology.orchestra.run.smithi165.stdout:5814 trunc from 0x79c5600 to 0xb242200 2024-09-17T21:48:44.225 INFO:teuthology.orchestra.run.smithi165.stdout:5817 read 0x35ed800 thru 0x35f9164 (0xb965 bytes) 2024-09-17T21:48:44.227 INFO:teuthology.orchestra.run.smithi165.stdout:5818 punch from 0xd56c00 to 0xd57200, (0x600 bytes) 2024-09-17T21:48:44.230 INFO:teuthology.orchestra.run.smithi165.stdout:5819 trunc from 0xb242200 to 0x884e200 2024-09-17T21:48:44.232 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:44.230+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:44.246 INFO:teuthology.orchestra.run.smithi165.stdout:5820 trunc from 0x884e200 to 0xc49d400 2024-09-17T21:48:44.258 INFO:teuthology.orchestra.run.smithi165.stdout:5821 trunc from 0xc49d400 to 0xe377800 2024-09-17T21:48:44.268 INFO:teuthology.orchestra.run.smithi165.stdout:5825 write 0xd265200 thru 0xd26b34e (0x614f bytes) 2024-09-17T21:48:44.269 INFO:teuthology.orchestra.run.smithi165.stdout:5830 write 0x55f8400 thru 0x560337d (0xaf7e bytes) 2024-09-17T21:48:44.275 INFO:teuthology.orchestra.run.smithi165.stdout:5833 trunc from 0xe377800 to 0x9496000 2024-09-17T21:48:44.296 INFO:teuthology.orchestra.run.smithi165.stdout:5834 read 0x8726800 thru 0x872e7e5 (0x7fe6 bytes) 2024-09-17T21:48:44.320 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 142.634152226, 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:44.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:44.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:44.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 142.60187725099999, 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.329 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:44.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 142.60163848299999, 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.335 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:44.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.434132352999995, 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:44.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.427230582000007, 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:44.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:44.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:44.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 93.427114490999998, 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:44.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:44.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.356 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.230110971000002, 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:44.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:44.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.230072176, 2024-09-17T21:48:44.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:44.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:44.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.230057891999998, 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:44.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:44.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.229812926000001, 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:44.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:44.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.229730478, 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:44.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:44.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.229698667999997, 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:44.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:44.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.229674949, 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:44.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:44.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.843035552000003, 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:44.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:44.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:44.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:44.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.833106571999998, 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:44.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:44.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:44.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:44.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.640771858000001, 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:44.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:44.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:44.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.640514437, 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:44.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:44.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.640481590999997, 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:44.415 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:44.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:44.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.640464645999998, 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:44.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:44.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.640423149, 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.434 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:44.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:44.437 INFO:teuthology.orchestra.run.smithi165.stdout:5836 write 0xcce5800 thru 0xccf2355 (0xcb56 bytes) 2024-09-17T21:48:44.437 INFO:teuthology.orchestra.run.smithi165.stdout:5840 write 0x62f5200 thru 0x6301e91 (0xcc92 bytes) 2024-09-17T21:48:44.437 INFO:teuthology.orchestra.run.smithi165.stdout:5843 write 0xbf67a00 thru 0xbf695e9 (0x1bea bytes) 2024-09-17T21:48:44.437 INFO:teuthology.orchestra.run.smithi165.stdout:5844 write 0xc7dc800 thru 0xc7eb5dd (0xedde bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5845 read 0x577d600 thru 0x5785111 (0x7b12 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5847 punch from 0xab99400 to 0xab9ac00, (0x1800 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5848 write 0xcacce00 thru 0xcad7fd9 (0xb1da bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5849 write 0xb2c5800 thru 0xb2d1335 (0xbb36 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5850 read 0xb022400 thru 0xb02f483 (0xd084 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5851 punch from 0xbce00 to 0xc6400, (0x9600 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5856 read 0x34c6a00 thru 0x34d4814 (0xde15 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5857 read 0x8629000 thru 0x862c133 (0x3134 bytes) 2024-09-17T21:48:44.438 INFO:teuthology.orchestra.run.smithi165.stdout:5858 clone 54 order 25 su 262144 sc 6 2024-09-17T21:48:44.438 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:44.433+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:44.536 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:44.534+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:44.584 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:44.584 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:44.584 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:44.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:44.637 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:44.634+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:44.740 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:44.735+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:44.741 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:44.741 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:44.741 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:44.748 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.3. 2024-09-17T21:48:44.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:48:44.838 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:44.836+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:44.849 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-17T21:48:44.863 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:44.863 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.777018755, 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.864 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:44.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:44.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775860422000001, 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:44.869 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:44.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:44.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:44.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775843877, 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775829111999997, 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:44.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775801807000001, 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:44.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:44.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775764918999997, 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:44.888 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:44.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:44.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775726220999999, 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.893 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:44.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775688047999999, 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:44.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:44.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775399708000002, 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:44.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:44.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.775373111999997, 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:44.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:44.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:44.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.200835306000002, 2024-09-17T21:48:44.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:44.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:44.919 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.200801990999999, 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.920 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:44.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.193711927000003, 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:44.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:44.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:44.931 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.193678247000001, 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:44.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.105209317, 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:44.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:48:44.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:48:44.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.105116298, 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:44.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:44.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:48:44.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:48:44.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 15.102486351, 2024-09-17T21:48:44.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:44.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:44.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.758943648000001, 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.957 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:48:44.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:48:44.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.103624151, 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:44.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:48:44.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:44.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:48:44.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 14.097715121, 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:48:44.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:48:44.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:48:44.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:48:44.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:44.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:44.974 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:44.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:44.976 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:44.973+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:45.076 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:45.073+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:45.124 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:45.125 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:45.125 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:45.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:45.176 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:45.174+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:45.277 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:45.274+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:45.280 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:45.280 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:45.280 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:45.280 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:45.280 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:45.290 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:45.378 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:45.376+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 130.52221690299999, 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:45.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:45.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:45.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:45.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:45.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485770592999998, 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485749292000001, 2024-09-17T21:48:45.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:45.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485728530999999, 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.413 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:45.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485707425000001, 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.418 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:45.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485686936, 2024-09-17T21:48:45.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:45.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:45.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485662839, 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:45.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485640345999997, 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:45.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485612738999997, 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.436 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.440 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 58.485587899000002, 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:45.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:45.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.333634124, 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:45.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.333610362000002, 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:45.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.333567973000001, 2024-09-17T21:48:45.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:45.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277819319999999, 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:45.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:45.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277796225000003, 2024-09-17T21:48:45.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:45.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:45.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277769845000002, 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:45.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277753208999997, 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:45.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:45.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277723244999997, 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:45.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277710319999997, 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:45.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.277700099, 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:45.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:45.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:45.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:45.489 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:45.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:45.491 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:45.488+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:45.591 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:45.588+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:45.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:45.611 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:45.611 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:45.611 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:45.620 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:45.704 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:45.702+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:45.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:45.751 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:45.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:45.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:45.767 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:45.770 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:45.770 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:45.770 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:45.779 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:45.784 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:45.785 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:45.793 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:45.790+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:45.801 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:45.801 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:45.818 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:45.818 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:45.835 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:48:45.893 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:45.891+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:45.929 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:45.929 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:45.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:45.995 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:45.992+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:46.044 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.045 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.053 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:46.096 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:46.093+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:46.196 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:46.194+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:46.201 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.201 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.209 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:46.297 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:46.295+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:46.356 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.357 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:46.397 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:46.396+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:46.472 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.472 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.480 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:46.498 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:46.496+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:46.586 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.586 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:46.638 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:46.636+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:46.700 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:46.698+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:46.701 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.701 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:46.801 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:46.798+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:46.831 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.831 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:46.902 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:46.900+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:46.966 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:46.966 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:46.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:47.003 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:47.001+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:47.059 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone53 from 0xccf2400 (overlap 0x1564000) to 0x88e600 2024-09-17T21:48:47.104 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:47.102+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:47.132 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.132 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.141 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:47.225 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:47.223+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:47.303 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.304 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.306 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:47.303+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:47.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:47.407 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:47.404+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:47.467 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.468 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:47.508 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:47.506+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:47.599 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.599 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.610 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:47.610 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:47.607+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:47.710 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:47.707+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:47.735 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.735 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:47.787 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #52, image image_client.0-clone52 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent53 2024-09-17T21:48:47.811 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:47.808+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:47.830 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:47.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:47.864 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:47.864 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:47.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:47.915 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:47.913+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:48.012 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:48.010+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:48.031 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:48.031 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:48.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:48.113 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:48.111+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:48.158 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:48.158 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:48.168 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:48.192 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:48.193 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:48.210 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:48.211 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:48.230 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:48.230 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:48.250 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:48.250 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:48.251 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:48.249+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:48.269 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:48.270 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:48.290 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:48.290 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:48.314 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:48.312+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:48.325 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:48.325 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:48.326 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:48.326 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:48.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:48.416 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:48.413+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:48.502 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:48.502 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 142.75959141600001, 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.503 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:48.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 142.758546956, 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.508 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:48.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:48.513 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.557361624999999, 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:48.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:48.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:48.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:48.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:48.520 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.341443957000003, 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:48.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:48.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.139007214999999, 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:48.525 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:48.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138950035000001, 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:48.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:48.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.538 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138379454000003, 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138351520000001, 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.544 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138217167999997, 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.549 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138195633000002, 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.554 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:48.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:48.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:48.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:48.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.138034668000003, 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.559 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:48.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.137978902999997, 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:48.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.136632296000002, 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:48.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:48.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.136580739999999, 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:48.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.135892478999999, 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:48.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:48.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.135875427999999, 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:48.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:48.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.134937188000002, 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:48.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:48.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.134906848, 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:48.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.840509400000002, 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:48.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:48.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 43.939391700000002, 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:48.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.612 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:48.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:48.614 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:48.611+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:48.665 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:48:48.723 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:48.721+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:48.751 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3.344c0dcf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.075101237000000001, 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.075125737999999997, 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:48:48.755 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:48.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:48.815 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:48.812+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:48.877 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:48.877 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:48.877 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:48.877 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:48.877 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:48.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:48.917 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:48.914+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:49.017 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:49.015+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:49.043 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:49.043 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.069150949, 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:49.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06912845599999, 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:49.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06910419600001, 2024-09-17T21:48:49.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:49.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06907831300001, 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:49.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:49.062 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06905331199999, 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:49.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:49.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06901351299999, 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:49.071 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.068997768, 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:49.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:49.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.068986233, 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:49.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06897593299999, 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:49.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06700469, 2024-09-17T21:48:49.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:49.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:49.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.092 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06699154200001, 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:49.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06698093599999, 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:49.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:49.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06697059800001, 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:49.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.066960591, 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:49.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.113 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:49.114 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06695063999999, 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:49.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:49.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:49.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.122 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06693533200001, 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.123 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:49.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:49.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.127 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11325 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1929216~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.064959445, 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068097668, 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11325 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975015+0000", 2024-09-17T21:48:49.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975016+0000", 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5600000000000002e-07 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975017+0000", 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:49.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975020+0000", 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3730000000000001e-06 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975023+0000", 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.711e-06 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042863+0000", 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067840229000000002 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042949+0000", 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6118000000000005e-05 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043113+0000", 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016365999999999999 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06494848899999, 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:49.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06493599999999, 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:49.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:49.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:49.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 152.06492553699999, 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:49.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.149 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:49.151 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:49.151 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:49.148+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:49.251 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:49.249+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:49.297 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3.1510611e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.62224998799999998, 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.62226968800000004, 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:49.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:49.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:48:49.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3.1510611e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.62166001599999998, 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.62172166100000004, 2024-09-17T21:48:49.302 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:48:49.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 2 2024-09-17T21:48:49.307 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:49.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:49.352 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:49.350+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:49.414 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:49.415 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:49.415 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:49.415 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:49.415 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:49.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:49.453 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:49.451+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:49.531 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:49.531 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3619 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5529600~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556414564000001, 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029125908999999998, 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3619 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:49.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971878+0000", 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4499999999999998e-07 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971880+0000", 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8950000000000001e-06 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971886+0000", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8760000000000002e-06 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971890+0000", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6160000000000004e-06 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000819+0000", 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.028928762 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.000896+0000", 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7145999999999996e-05 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001003+0000", 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010726900000000001 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556396542, 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.537 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:48:49.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:48:49.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556372404999999, 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:48:49.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.55634015, 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556310097000001, 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:48:49.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556286751, 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:49.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:48:49.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556233804, 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:48:49.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:48:49.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.556200457999999, 2024-09-17T21:48:49.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:49.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3975 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5120000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:49.571 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552892865, 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029182326000000001, 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3975 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975399+0000", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975400+0000", 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:49.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975402+0000", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8840000000000001e-06 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975407+0000", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975412+0000", 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4689999999999999e-06 2024-09-17T21:48:49.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004419+0000", 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029007848999999999 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004494+0000", 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4351999999999996e-05 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004582+0000", 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7769000000000006e-05 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552869225, 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552841324999999, 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:48:49.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552818619, 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:48:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:49.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552802351, 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.55279238, 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:48:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:48:49.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552780231, 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.600 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552770536000001, 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:48:49.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552760942999999, 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:48:49.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:49.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552751452999999, 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:48:49.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552741607, 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:49.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.552732267, 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:49.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:48:49.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:49.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:49.628 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:49.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:49.630 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:49.627+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:49.730 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:49.727+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:49.759 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:49.759 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:49.760 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:49.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:49.830 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:49.828+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:49.849 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 3] out_osds: [2] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:48:49.849 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:48:49.850 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [1, 5, 4, 0, 3] 2024-09-17T21:48:49.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-17T21:48:49.877 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:49.877 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:49.877 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:49.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:49.970 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:49.968+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:50.032 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:50.029+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:50.040 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 148.35400489099999, 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:50.041 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:50.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 148.32172991600001, 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:50.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:50.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 148.32149114800001, 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:50.053 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:50.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:50.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.153985018, 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:50.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:50.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:50.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:50.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.147083246999998, 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:50.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:50.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.146967156000002, 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:50.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:50.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949963636000007, 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:50.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949924840999998, 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:50.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949910556999995, 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.085 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:50.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:50.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:50.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949665590999999, 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:50.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949583142999998, 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:50.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949551333000002, 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:50.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 64.949527614000004, 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:50.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:50.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.562888217000001, 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:50.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:50.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:50.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.552959237000003, 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:50.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.360624522999998, 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:50.121 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:50.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:50.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:50.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.360367101999998, 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:50.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.360334256000002, 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.134 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:50.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.360317311000003, 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:50.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:50.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 42.360275813999998, 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:50.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:50.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:50.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:50.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:50.153 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:50.155 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:50.155 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:48:50.151+0000 7f0c28614640 -1 mon.a@0(leader).osd e167 definitely_dead 0 2024-09-17T21:48:50.155 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:50.152+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:50.255 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:50.253+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:50.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:50.302 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:50.303 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:50.303 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:50.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:50.356 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:50.353+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:50.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:50.426 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:50.443 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:50.443 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:50.457 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:50.454+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:50.459 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:50.464 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:50.465 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:50.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:50.478 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:50.478 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:50.496 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:50.514 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:50.514 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:50.558 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:50.555+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:50.578 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:50.578 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:50.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:50.658 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:50.656+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:50.665 INFO:teuthology.orchestra.run.smithi007.stderr:marked out osd.3. 2024-09-17T21:48:50.682 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:48:50.699 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:50.700 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:50.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:50.759 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:50.756+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:50.858 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:50.858 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:50.859 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:50.857+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:50.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:50.960 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:50.958+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:51.016 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.017 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:51.066 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:51.064+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:51.146 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.146 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:51.162 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:51.160+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:51.262 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:51.260+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:51.263 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.263 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:51.363 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:51.361+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:51.418 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.418 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.427 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:51.471 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:51.469+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:51.534 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.534 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.542 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:51.584 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:51.581+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:51.666 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:51.664+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:51.667 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:48:51.709 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.709 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:51.767 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:51.765+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:51.867 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.867 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.868 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:51.865+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:51.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:51.969 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:51.967+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:51.983 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:51.983 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:51.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:52.069 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:52.067+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:52.099 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:52.099 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:52.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:52.170 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:52.167+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:52.255 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:52.255 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:52.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:52.307 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:52.305+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:52.372 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:52.370+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:52.384 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:52.384 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:52.393 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:52.473 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:52.471+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:52.526 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:52.541 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:52.541 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:52.551 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:52.591 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:52.589+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:52.650 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:52.650 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:52.667 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:52.668 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:52.675 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:52.672+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:52.702 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:52.702 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:52.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 151.03838000100001, 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 151.006105026, 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 151.005866258, 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.83836012800001, 2024-09-17T21:48:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.831458357, 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.83134226600001, 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.634338745999997, 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.634299951000003, 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.634285667, 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.634040701000004, 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.633958253000003, 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.633926443000007, 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12792 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.633902723999995, 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769545399999997, 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12792 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087298+0000", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087299+0000", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087301+0000", 2024-09-17T21:48:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1940000000000001e-06 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087308+0000", 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7519999999999999e-06 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087313+0000", 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4380000000000003e-06 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087367+0000", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3810000000000001e-05 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087373+0000", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7869999999999998e-06 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004986+0000", 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.917612859 2024-09-17T21:48:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004994+0000", 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2369999999999992e-06 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.247263326999999, 2024-09-17T21:48:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.237334347000001, 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.044999633000003, 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:52.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.044742212000003, 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.044709365999999, 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:52.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.044692421000001, 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.044650924000003, 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:52.837 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:52.838 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:52.838 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:52.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:52.840 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:52.840 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:52.840 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:52.841 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:52.842 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:52.836+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:52.939 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:52.937+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:53.040 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:53.037+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:53.040 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6427.0:2874 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [sparse-read 7733248~0 out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e170)", 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.038447+0000", 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.000255973, 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00028436200000000003, 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6427", 2024-09-17T21:48:53.041 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2874 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038447+0000", 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038447+0000", 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038448+0000", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7400000000000002e-07 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038452+0000", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3610000000000003e-06 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038462+0000", 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0261000000000001e-05 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038470+0000", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3719999999999999e-06 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038533+0000", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3649e-05 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.038641+0000", 2024-09-17T21:48:53.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000108 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:48:53.045 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:53.055 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:53.141 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:53.139+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:53.227 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:53.227 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:53.227 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:53.227 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:53.227 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:53.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:53.280 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:53.278+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:53.331 INFO:teuthology.orchestra.run.smithi165.stdout:5859 write 0x3a22200 thru 0x3a2c27f (0xa080 bytes) 2024-09-17T21:48:53.333 INFO:teuthology.orchestra.run.smithi165.stdout:5862 read 0xa6c0a00 thru 0xa6cbf38 (0xb539 bytes) 2024-09-17T21:48:53.334 INFO:teuthology.orchestra.run.smithi165.stdout:5863 read 0x9b7f000 thru 0x9b8b16d (0xc16e bytes) 2024-09-17T21:48:53.334 INFO:teuthology.orchestra.run.smithi165.stdout:5864 read 0x3ec0800 thru 0x3ec8ef8 (0x86f9 bytes) 2024-09-17T21:48:53.335 INFO:teuthology.orchestra.run.smithi165.stdout:5865 write 0x74f4600 thru 0x74fe2e4 (0x9ce5 bytes) 2024-09-17T21:48:53.336 INFO:teuthology.orchestra.run.smithi165.stdout:5866 read 0xb03d400 thru 0xb0463d2 (0x8fd3 bytes) 2024-09-17T21:48:53.337 INFO:teuthology.orchestra.run.smithi165.stdout:5868 write 0x12bf200 thru 0x12c6619 (0x741a bytes) 2024-09-17T21:48:53.339 INFO:teuthology.orchestra.run.smithi165.stdout:5870 punch from 0x7ad600 to 0x7ae000, (0xa00 bytes) 2024-09-17T21:48:53.342 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:53.340+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:53.441 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:53.441 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:53.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.353622383000001, 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:53.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:53.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352464050000002, 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.447 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:53.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:53.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352447505000001, 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.452 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:53.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352432739999998, 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.457 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:53.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352405435000001, 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:53.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:53.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352368546999998, 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:53.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352329849, 2024-09-17T21:48:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352291676, 2024-09-17T21:48:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.352003336000003, 2024-09-17T21:48:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.351976739999998, 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:53.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.777438934000003, 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:53.492 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:53.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.777405619, 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:53.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:53.503 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.770315555000003, 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:53.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.770281875000002, 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:53.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:53.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.681812945000001, 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:53.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:48:53.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.681719926, 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.521 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:53.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:53.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.679089979, 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:53.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:53.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:53.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:48:53.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:48:53.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 23.335547276, 2024-09-17T21:48:53.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:48:53.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:53.542 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.680227778999999, 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:53.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:48:53.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:53.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 22.674318749000001, 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:48:53.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:53.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:53.555 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:53.556 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:53.557 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:53.554+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:53.657 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:53.654+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:53.671 INFO:teuthology.orchestra.run.smithi165.stdout:5871 read 0xc1bcc00 thru 0xc1c60d7 (0x94d8 bytes) 2024-09-17T21:48:53.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:48:53.733 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:53.734 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:53.734 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:53.734 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:53.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:53.758 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:53.755+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:53.858 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:53.858 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:53.858 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:53.858 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:53.858 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:53.859 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:53.856+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:53.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:53.960 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:53.957+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:54.016 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:54.016 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.140392859, 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:54.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:48:54.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:54.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103946549, 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:48:54.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:48:54.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103925247999996, 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.028 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:48:54.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103904486999994, 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:48:54.032 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:48:54.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:48:54.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103883381000003, 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:48:54.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:48:54.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:48:54.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103862891999995, 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.042 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:48:54.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:54.046 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103838795000001, 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103816301999998, 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:48:54.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103788695000006, 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.056 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.103763854999997, 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:48:54.060 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:48:54.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.951810080000001, 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.951786318000003, 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.073 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.951743929000003, 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:48:54.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:48:54.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:48:54.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895995276000001, 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:48:54.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:48:54.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895972180999998, 2024-09-17T21:48:54.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:54.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:48:54.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895945801000003, 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:48:54.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:54.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895929164999998, 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895899200999999, 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:48:54.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895886275999999, 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:48:54.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 62.895876055000002, 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:54.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:54.109 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:54.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:54.111 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:54.108+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:54.211 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:54.209+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:54.222 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:54.222 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:54.223 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:54.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:54.313 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:54.310+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:54.337 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:54.337 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:54.337 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:54.346 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:54.413 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:54.411+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:54.495 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 148.752659733, 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:48:54.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 148.751615273, 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:48:54.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:48:54.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:48:54.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.550429941999994, 2024-09-17T21:48:54.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:48:54.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:48:54.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:54.513 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.334512273999998, 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:48:54.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.132075532000002, 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:48:54.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.132018352000003, 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:48:54.525 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:48:54.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:48:54.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:54.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131447770999998, 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:48:54.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:48:54.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:48:54.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131419837000003, 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:48:54.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131285484999999, 2024-09-17T21:48:54.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:54.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131263949999997, 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:54.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:48:54.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131102984999998, 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:48:54.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.131047219999999, 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:48:54.562 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.129700612999997, 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.563 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:48:54.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.129649057000002, 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.128960796000001, 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:48:54.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.128943745000001, 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:48:54.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.128005504999997, 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:48:54.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:48:54.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.588 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.127975165000002, 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:48:54.589 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:48:54.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.833577716999997, 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:48:54.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.932460016999997, 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:48:54.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:48:54.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:48:54.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:48:54.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.605 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:54.606 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:54.606 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:54.603+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:54.668 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:48:54.676 INFO:teuthology.orchestra.run.smithi165.stdout:5873 read 0x7bc4800 thru 0x7bd0094 (0xb895 bytes) 2024-09-17T21:48:54.677 INFO:teuthology.orchestra.run.smithi165.stdout:5874 write 0xc07ce00 thru 0xc088d96 (0xbf97 bytes) 2024-09-17T21:48:54.729 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:54.727+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6474.0:4110 3.16 3.f1564396 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e172)", 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:54.676150+0000", 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.094104082000000006, 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.094126037999999995, 2024-09-17T21:48:54.772 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6474", 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 4110 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676150+0000", 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676150+0000", 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676151+0000", 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.421e-06 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:54.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676157+0000", 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5969999999999999e-06 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676177+0000", 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0370999999999999e-05 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:54.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676187+0000", 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7540000000000001e-06 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676484+0000", 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029656099999999998 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:54.676497+0000", 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3032e-05 2024-09-17T21:48:54.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:48:54.777 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:54.781 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:54.824 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:54.822+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:54.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:54.889 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:54.889 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:54.889 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:54.889 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:54.889 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:54.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:54.940 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:54.937+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:54.975 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:54.976 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:54.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:54.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:55.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:55.009 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:55.007+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:55.025 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:55.025 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:55.042 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:55.042 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07399181700001, 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:55.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07396932399999, 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:55.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07394506399999, 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:55.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07391918100001, 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:55.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07389418, 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:55.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.071 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07385438099999, 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.073838636, 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:55.076 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:55.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07382710100001, 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:55.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:55.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07381680099999, 2024-09-17T21:48:55.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:55.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07184555800001, 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:55.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07183241000001, 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:55.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.071821804, 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:55.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:55.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07181146600001, 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.071801459, 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.108 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:55.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07179150799999, 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.112 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:55.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:55.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:55.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.118 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:55.119 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.07177619999999, 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.120 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:55.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:55.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:55.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:55.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:55.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:55.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.06978935699999, 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.131 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:55.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:55.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:55.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:55.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:55.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:55.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:55.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.140 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.06977686799999, 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.141 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.142 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:55.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:55.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.146 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 158.069766405, 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:55.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.3704306070000003, 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:55.152 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:48:55.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:48:55.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:55.157 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:55.158 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:55.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:55.160 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:55.160 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:55.156+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:55.259 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:55.257+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:55.267 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.267 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.275 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:55.361 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:55.359+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:55.382 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.382 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.391 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:55.461 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:55.459+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:55.497 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.497 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:55.562 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:55.560+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:55.618 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.618 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:55.671 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:55.669+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:55.682 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:48:55.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:48:55.691 INFO:teuthology.orchestra.run.smithi165.stdout:5876 trunc from 0xccf2400 to 0x4578c00 2024-09-17T21:48:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.759 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.764 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:55.761+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:55.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:55.865 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:55.863+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:55.880 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.880 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:55.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:55.966 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:55.964+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:55.997 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.006 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:56.067 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:56.065+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:56.099 INFO:teuthology.orchestra.run.smithi165.stdout:5878 write 0xe87c400 thru 0xe88743d (0xb03e bytes) 2024-09-17T21:48:56.101 INFO:teuthology.orchestra.run.smithi165.stdout:5879 write 0x3f90e00 thru 0x3f94ef0 (0x40f1 bytes) 2024-09-17T21:48:56.101 INFO:teuthology.orchestra.run.smithi165.stdout:5880 read 0x4898e00 thru 0x4899744 (0x945 bytes) 2024-09-17T21:48:56.113 INFO:teuthology.orchestra.run.smithi165.stdout:5882 trunc from 0xe887600 to 0x6841a00 2024-09-17T21:48:56.156 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.156 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.157 INFO:teuthology.orchestra.run.smithi165.stdout:5883 write 0x78b3800 thru 0x78b49c2 (0x11c3 bytes) 2024-09-17T21:48:56.158 INFO:teuthology.orchestra.run.smithi165.stdout:5884 read 0x4f73000 thru 0x4f81820 (0xe821 bytes) 2024-09-17T21:48:56.158 INFO:teuthology.orchestra.run.smithi165.stdout:5887 read 0x64db600 thru 0x64ead47 (0xf748 bytes) 2024-09-17T21:48:56.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:56.168 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:56.165+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:56.268 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:56.266+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:56.272 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.272 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:56.369 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:56.367+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:56.387 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.387 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.397 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:56.423 INFO:teuthology.orchestra.run.smithi165.stdout:5888 write 0x9467c00 thru 0x9471fcd (0xa3ce bytes) 2024-09-17T21:48:56.424 INFO:teuthology.orchestra.run.smithi165.stdout:5890 punch from 0x68c1200 to 0x68cf000, (0xde00 bytes) 2024-09-17T21:48:56.424 INFO:teuthology.orchestra.run.smithi165.stdout:5891 trunc from 0x9472000 to 0x9ad1a00 2024-09-17T21:48:56.448 INFO:teuthology.orchestra.run.smithi165.stdout:5892 read 0x563400 thru 0x5703ff (0xd000 bytes) 2024-09-17T21:48:56.448 INFO:teuthology.orchestra.run.smithi165.stdout:5895 clone 55 order 23 su 65536 sc 15 2024-09-17T21:48:56.471 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:56.468+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:56.504 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.504 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:56.571 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:56.569+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:56.619 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.620 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:56.672 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:56.669+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:56.688 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:48:56.737 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.737 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.745 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:56.773 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:56.771+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:56.863 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:48:56.874 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:56.871+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:56.878 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:48:56.896 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:56.897 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:56.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:56.975 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:56.972+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:57.016 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:57.016 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:57.025 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:48:57.057 INFO:teuthology.orchestra.run.smithi165.stdout:truncating image image_client.0-clone54 from 0x9ad1a00 (overlap 0x4578c00) to 0x0 2024-09-17T21:48:57.076 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:57.074+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:57.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:48:57.178 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:57.178 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:57.187 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:48:57.204 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:57.202+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:57.278 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:57.275+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:57.287 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:57.287 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:57.297 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:57.297 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:57.304 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:57.304 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:57.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:48:57.321 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:57.321 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:57.338 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:57.339 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:57.356 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:57.356 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:57.372 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:57.372 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:57.378 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:57.376+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11308 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1093632~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44293187599999, 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068261358999999994, 2024-09-17T21:48:57.417 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11308 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970824+0000", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1100000000000001e-07 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970827+0000", 2024-09-17T21:48:57.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.199e-06 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970834+0000", 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4830000000000002e-06 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970839+0000", 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3199999999999999e-06 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038906+0000", 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068066266 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.038968+0000", 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2113000000000007e-05 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039085+0000", 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011756699999999999 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.442909383, 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:48:57.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:48:57.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.442885123, 2024-09-17T21:48:57.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44285923999999, 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.435 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44283423900001, 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:48:57.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:48:57.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44279444, 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:48:57.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:48:57.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44277869499999, 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:48:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:48:57.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44276715999999, 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:48:57.449 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:48:57.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.453 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44275686, 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:48:57.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:57.458 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44078561699999, 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:57.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:48:57.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.440772469, 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:48:57.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44076186300001, 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:48:57.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:48:57.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.440751525, 2024-09-17T21:48:57.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:48:57.476 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.44074151800001, 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.477 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:48:57.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.481 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.440731567, 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:48:57.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:48:57.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.440716259, 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:48:57.486 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:48:57.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.438729416, 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:48:57.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.438716927, 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:48:57.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.43870646400001, 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:48:57.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:48:57.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7393706659999992, 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:48:57.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:48:57.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:48:57.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.511 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:57.513 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:48:57.513 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:48:57.510+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:48:57.613 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:57.611+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:57.661 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:57.661 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:57.661 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:48:57.661 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:57.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:48:57.753 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:57.751+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:57.815 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:57.813+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:57.822 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:57.822 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:48:57.822 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:48:57.822 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:48:57.822 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:57.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:48:57.867 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:48:57.868 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:48:57.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:48:57.925 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:57.923+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:57.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006986654999999, 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:48:57.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006962518000002, 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:48:57.987 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:48:57.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006930263000001, 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:57.992 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:48:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:57.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006900210000001, 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:57.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:57.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:57.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:48:58.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006876863999999, 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:48:58.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:48:58.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.006 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006823916999998, 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:48:58.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:48:58.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.006790571, 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:48:58.011 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:48:58.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003459337999999, 2024-09-17T21:48:58.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:48:58.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:48:58.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.021 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003431438, 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:58.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003408732, 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.029 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:48:58.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003392464000001, 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:48:58.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:48:58.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003382493, 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:48:58.038 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:48:58.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003370344, 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:48:58.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003360649000001, 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:58.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:48:58.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:48:58.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003351056, 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.056 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003341566, 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:48:58.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:48:58.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.003331719999998, 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.061 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:48:58.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:48:58.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.00332238, 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:48:58.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.070 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.3051089660000006, 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:48:58.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.3045189940000004, 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:48:58.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:48:58.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:48:58.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:48:58.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:48:58.081 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:48:58.082 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:58.080+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:58.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:48:58.183 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:58.181+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:58.191 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:58.191 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:58.191 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:58.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:48:58.284 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:58.281+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:58.347 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:58.347 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:58.347 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:58.348 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:58.348 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:58.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:48:58.399 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:58.397+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 156.77858496299999, 2024-09-17T21:48:58.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:58.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:48:58.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:58.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 156.74630998800001, 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:48:58.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 156.74607122, 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:48:58.477 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:48:58.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:48:58.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.57856509, 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.482 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:48:58.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:48:58.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.571663319, 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.488 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:48:58.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.571547228, 2024-09-17T21:48:58.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:58.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:48:58.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374543708000004, 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:48:58.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374504912999996, 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:48:58.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374490628999993, 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:48:58.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374245662999996, 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:48:58.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:48:58.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:48:58.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374163214999996, 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:48:58.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:48:58.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.374131405, 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.524 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.987468289000006, 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:48:58.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:48:58.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:48:58.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:58.534 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.977539308999994, 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:48:58.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.785204595000003, 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:58.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:48:58.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.784947174000003, 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:48:58.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:48:58.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.784914327999999, 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:48:58.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:48:58.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.559 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.784897383000001, 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:48:58.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.562 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 50.784855886000003, 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:48:58.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:48:58.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 4.7906193830000001, 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:58.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:48:58.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:58.577 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:58.578 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:48:58.579 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:58.576+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:58.668 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:48:58.679 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:58.677+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:58.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:48:58.730 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:58.730 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:58.730 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:58.730 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:58.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:48:58.780 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:58.778+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:58.881 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:58.878+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:58.889 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:58.889 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:58.889 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:58.890 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:58.890 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:58.898 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:48:58.982 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:48:58.980+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.920820312000004, 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.008 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:48:59.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919661978999997, 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.013 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:48:59.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919645434000003, 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.018 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:48:59.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919630669, 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.023 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:48:59.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:48:59.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919603363999997, 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.028 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:48:59.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:48:59.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:59.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919566476, 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:59.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:48:59.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:48:59.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919527778000003, 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:48:59.038 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:48:59.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:48:59.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:59.043 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919489605000003, 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:59.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:48:59.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:48:59.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:59.048 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919201264999998, 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:59.049 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.050 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.051 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:48:59.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.919174669, 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:48:59.053 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.054 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:48:59.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.344636862999998, 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.059 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:48:59.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:48:59.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:48:59.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.344603548000002, 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.064 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:48:59.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:48:59.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.337513483999999, 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:48:59.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:48:59.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.337479803999997, 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:48:59.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:48:59.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.249010874, 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:59.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:48:59.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:48:59.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.248917854999998, 2024-09-17T21:48:59.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:48:59.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 29.246287907999999, 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:48:59.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:48:59.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:48:59.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:48:59.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:48:59.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.902745204999999, 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:48:59.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:48:59.110 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.247425708000002, 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:48:59.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:59.118 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 28.241516678, 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:48:59.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:48:59.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:48:59.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:48:59.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:48:59.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:48:59.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:48:59.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:48:59.128 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:59.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:48:59.130 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:48:59.127+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:48:59.230 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:59.228+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:59.277 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:59.277 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:59.277 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:48:59.277 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:59.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:48:59.331 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:59.329+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:59.384 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:48:59.393 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:48:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:48:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:48:59.393 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:48:59.393 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:48:59.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:48:59.443 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:59.441+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:59.509 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:48:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:48:59.527 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:48:59.527 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:48:59.533 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:48:59.531+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:48:59.543 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:48:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:48:59.553 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:59.553 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:59.561 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:48:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:48:59.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:48:59.578 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:48:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:48:59.595 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:48:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:48:59.647 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:59.645+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:48:59.680 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:48:59.681 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:48:59.682 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:59.682 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:59.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:48:59.735 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:59.733+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:59.799 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:59.799 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:59.807 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:48:59.837 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #53, image image_client.0-clone53 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent54 2024-09-17T21:48:59.848 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:48:59.846+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:48:59.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:48:59.937 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:48:59.935+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:48:59.952 INFO:teuthology.orchestra.run.smithi165.stdout:5897 write 0x262d400 thru 0x263135f (0x3f60 bytes) 2024-09-17T21:48:59.953 INFO:teuthology.orchestra.run.smithi165.stdout:5898 read 0x5110400 thru 0x5112f03 (0x2b04 bytes) 2024-09-17T21:48:59.954 INFO:teuthology.orchestra.run.smithi165.stdout:5899 write 0x453a00 thru 0x4552b8 (0x18b9 bytes) 2024-09-17T21:48:59.955 INFO:teuthology.orchestra.run.smithi165.stdout:5900 read 0x3d7da00 thru 0x3d7f119 (0x171a bytes) 2024-09-17T21:48:59.956 INFO:teuthology.orchestra.run.smithi165.stdout:5901 read 0xdec200 thru 0xdf6c4a (0xaa4b bytes) 2024-09-17T21:48:59.957 INFO:teuthology.orchestra.run.smithi165.stdout:5902 read 0x838c00 thru 0x83f0be (0x64bf bytes) 2024-09-17T21:48:59.958 INFO:teuthology.orchestra.run.smithi165.stdout:5903 write 0x3454400 thru 0x345f8db (0xb4dc bytes) 2024-09-17T21:48:59.958 INFO:teuthology.orchestra.run.smithi165.stdout:5904 write 0x29b000 thru 0x2a330d (0x830e bytes) 2024-09-17T21:48:59.959 INFO:teuthology.orchestra.run.smithi165.stdout:5905 write 0x3da3a00 thru 0x3dad05c (0x965d bytes) 2024-09-17T21:48:59.960 INFO:teuthology.orchestra.run.smithi165.stdout:5907 read 0x478a400 thru 0x478cd5d (0x295e bytes) 2024-09-17T21:48:59.961 INFO:teuthology.orchestra.run.smithi165.stdout:5908 write 0x3682600 thru 0x368adab (0x87ac bytes) 2024-09-17T21:48:59.961 INFO:teuthology.orchestra.run.smithi165.stdout:5909 punch from 0x26c9600 to 0x26d6c00, (0xd600 bytes) 2024-09-17T21:48:59.973 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:48:59.973 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:48:59.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:00.037 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:00.035+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:00.058 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:00.058 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:49:00.072 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-17T21:48:59.664714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2114813046,"num_objects":2381,"num_object_clones":468,"num_object_copies":4762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2381,"num_whiteouts":1030,"num_read":356950,"num_read_kb":1803467,"num_write":19842,"num_write_kb":586402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8193,"num_bytes_recovered":976611578,"num_keys_recovered":1680,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17176,"ondisk_log_size":17176,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1111904,"kb_used_data":882392,"kb_used_omap":6,"kb_used_meta":223225,"kb_avail":376375456,"statfs":{"total":386547056640,"available":385408466944,"internally_reserved":0,"allocated":903569408,"data_stored":1247895667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6380,"internal_metadata":228583188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90970702,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4317,"num_read_kb":38712,"num_write":480,"num_write_kb":52112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":64066528,"num_keys_recovered":95,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.996450"},"pg_stats":[{"pgid":"3.1f","version":"153'321","reported_seq":10779,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695583+0000","last_change":"2024-09-17T21:48:54.673693+0000","last_active":"2024-09-17T21:48:55.695583+0000","last_peered":"2024-09-17T21:48:55.695583+0000","last_clean":"2024-09-17T21:48:55.695583+0000","last_became_active":"2024-09-17T21:48:54.672690+0000","last_became_peered":"2024-09-17T21:48:54.672690+0000","last_unstale":"2024-09-17T21:48:55.695583+0000","last_undegraded":"2024-09-17T21:48:55.695583+0000","last_fullsized":"2024-09-17T21:48:55.695583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:19.152600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":60983108,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":34,"num_read":11050,"num_read_kb":70720,"num_write":309,"num_write_kb":11933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":50607226,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'487","reported_seq":19659,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665251+0000","last_change":"2024-09-17T21:48:54.675720+0000","last_active":"2024-09-17T21:48:58.665251+0000","last_peered":"2024-09-17T21:48:58.665251+0000","last_clean":"2024-09-17T21:48:58.665251+0000","last_became_active":"2024-09-17T21:48:54.674393+0000","last_became_peered":"2024-09-17T21:48:54.674393+0000","last_unstale":"2024-09-17T21:48:58.665251+0000","last_undegraded":"2024-09-17T21:48:58.665251+0000","last_fullsized":"2024-09-17T21:48:58.665251+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:02.807770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":115875888,"num_objects":97,"num_object_clones":26,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":41,"num_read":18708,"num_read_kb":109022,"num_write":393,"num_write_kb":34487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":559,"num_bytes_recovered":109255344,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"173'327","reported_seq":4196,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.127275+0000","last_change":"2024-09-17T21:47:10.889674+0000","last_active":"2024-09-17T21:48:56.127275+0000","last_peered":"2024-09-17T21:48:56.127275+0000","last_clean":"2024-09-17T21:48:56.127275+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:48:56.127275+0000","last_undegraded":"2024-09-17T21:48:56.127275+0000","last_fullsized":"2024-09-17T21:48:56.127275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:34.713443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":80021685,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":35,"num_read":4018,"num_read_kb":38346,"num_write":603,"num_write_kb":31640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"164'482","reported_seq":17394,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670347+0000","last_change":"2024-09-17T21:47:11.890576+0000","last_active":"2024-09-17T21:48:55.670347+0000","last_peered":"2024-09-17T21:48:55.670347+0000","last_clean":"2024-09-17T21:48:55.670347+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:48:55.670347+0000","last_undegraded":"2024-09-17T21:48:55.670347+0000","last_fullsized":"2024-09-17T21:48:55.670347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:59.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":88333502,"num_objects":88,"num_object_clones":22,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":31,"num_read":18554,"num_read_kb":85866,"num_write":533,"num_write_kb":39738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"164'410","reported_seq":16479,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.751148+0000","last_change":"2024-09-17T21:48:08.676407+0000","last_active":"2024-09-17T21:48:55.751148+0000","last_peered":"2024-09-17T21:48:55.751148+0000","last_clean":"2024-09-17T21:48:55.751148+0000","last_became_active":"2024-09-17T21:48:08.675352+0000","last_became_peered":"2024-09-17T21:48:08.675352+0000","last_unstale":"2024-09-17T21:48:55.751148+0000","last_undegraded":"2024-09-17T21:48:55.751148+0000","last_fullsized":"2024-09-17T21:48:55.751148+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:15.599106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":70805726,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":25,"num_read":14331,"num_read_kb":42652,"num_write":341,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"164'574","reported_seq":10593,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670715+0000","last_change":"2024-09-17T21:47:10.889765+0000","last_active":"2024-09-17T21:48:54.670715+0000","last_peered":"2024-09-17T21:48:54.670715+0000","last_clean":"2024-09-17T21:48:54.670715+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:48:54.670715+0000","last_undegraded":"2024-09-17T21:48:54.670715+0000","last_fullsized":"2024-09-17T21:48:54.670715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:22.524109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":101254943,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":31,"num_read":12545,"num_read_kb":80145,"num_write":627,"num_write_kb":25531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'364","reported_seq":4535,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664453+0000","last_change":"2024-09-17T21:47:11.890115+0000","last_active":"2024-09-17T21:48:58.664453+0000","last_peered":"2024-09-17T21:48:58.664453+0000","last_clean":"2024-09-17T21:48:58.664453+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:48:58.664453+0000","last_undegraded":"2024-09-17T21:48:58.664453+0000","last_fullsized":"2024-09-17T21:48:58.664453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:01.508494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":47057042,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4319,"num_read_kb":31678,"num_write":368,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"173'601","reported_seq":12627,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.116154+0000","last_change":"2024-09-17T21:48:08.686643+0000","last_active":"2024-09-17T21:48:56.116154+0000","last_peered":"2024-09-17T21:48:56.116154+0000","last_clean":"2024-09-17T21:48:56.116154+0000","last_became_active":"2024-09-17T21:48:08.681464+0000","last_became_peered":"2024-09-17T21:48:08.681464+0000","last_unstale":"2024-09-17T21:48:56.116154+0000","last_undegraded":"2024-09-17T21:48:56.116154+0000","last_fullsized":"2024-09-17T21:48:56.116154+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:57.602128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":90423561,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":11977,"num_read_kb":67143,"num_write":726,"num_write_kb":13790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":9646917,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"154'315","reported_seq":7169,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695481+0000","last_change":"2024-09-17T21:48:10.693039+0000","last_active":"2024-09-17T21:48:55.695481+0000","last_peered":"2024-09-17T21:48:55.695481+0000","last_clean":"2024-09-17T21:48:55.695481+0000","last_became_active":"2024-09-17T21:48:10.691486+0000","last_became_peered":"2024-09-17T21:48:10.691486+0000","last_unstale":"2024-09-17T21:48:55.695481+0000","last_undegraded":"2024-09-17T21:48:55.695481+0000","last_fullsized":"2024-09-17T21:48:55.695481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:43.032485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":19967504,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":34,"num_read":7364,"num_read_kb":30958,"num_write":302,"num_write_kb":8410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"170'866","reported_seq":10844,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.726454+0000","last_change":"2024-09-17T21:48:55.672954+0000","last_active":"2024-09-17T21:48:55.726454+0000","last_peered":"2024-09-17T21:48:55.726454+0000","last_clean":"2024-09-17T21:48:55.726454+0000","last_became_active":"2024-09-17T21:48:55.671674+0000","last_became_peered":"2024-09-17T21:48:55.671674+0000","last_unstale":"2024-09-17T21:48:55.726454+0000","last_undegraded":"2024-09-17T21:48:55.726454+0000","last_fullsized":"2024-09-17T21:48:55.726454+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:54.618156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":63860296,"num_objects":90,"num_object_clones":20,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":47,"num_read":13380,"num_read_kb":73472,"num_write":946,"num_write_kb":17143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":46911229,"num_keys_recovered":153,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'642","reported_seq":9385,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.748304+0000","last_change":"2024-09-17T21:48:51.685446+0000","last_active":"2024-09-17T21:48:55.748304+0000","last_peered":"2024-09-17T21:48:55.748304+0000","last_clean":"2024-09-17T21:48:55.748304+0000","last_became_active":"2024-09-17T21:48:51.675643+0000","last_became_peered":"2024-09-17T21:48:51.675643+0000","last_unstale":"2024-09-17T21:48:55.748304+0000","last_undegraded":"2024-09-17T21:48:55.748304+0000","last_fullsized":"2024-09-17T21:48:55.748304+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:21.493584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":53377608,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":16335,"num_read_kb":58528,"num_write":637,"num_write_kb":14179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":33289947,"num_keys_recovered":140,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'333","reported_seq":17669,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670049+0000","last_change":"2024-09-17T21:47:17.916575+0000","last_active":"2024-09-17T21:48:54.670049+0000","last_peered":"2024-09-17T21:48:54.670049+0000","last_clean":"2024-09-17T21:48:54.670049+0000","last_became_active":"2024-09-17T21:47:17.915249+0000","last_became_peered":"2024-09-17T21:47:17.915249+0000","last_unstale":"2024-09-17T21:48:54.670049+0000","last_undegraded":"2024-09-17T21:48:54.670049+0000","last_fullsized":"2024-09-17T21:48:54.670049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:21.536664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":146670737,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":28,"num_read":16461,"num_read_kb":133829,"num_write":424,"num_write_kb":41720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":39957736,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":276,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670305+0000","last_change":"2024-09-17T21:48:51.674495+0000","last_active":"2024-09-17T21:48:54.670305+0000","last_peered":"2024-09-17T21:48:54.670305+0000","last_clean":"2024-09-17T21:48:54.670305+0000","last_became_active":"2024-09-17T21:48:51.674180+0000","last_became_peered":"2024-09-17T21:48:51.674180+0000","last_unstale":"2024-09-17T21:48:54.670305+0000","last_undegraded":"2024-09-17T21:48:54.670305+0000","last_fullsized":"2024-09-17T21:48:54.670305+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:03.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"164'566","reported_seq":9672,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.755865+0000","last_change":"2024-09-17T21:48:10.693505+0000","last_active":"2024-09-17T21:48:55.755865+0000","last_peered":"2024-09-17T21:48:55.755865+0000","last_clean":"2024-09-17T21:48:55.755865+0000","last_became_active":"2024-09-17T21:48:10.691607+0000","last_became_peered":"2024-09-17T21:48:10.691607+0000","last_unstale":"2024-09-17T21:48:55.755865+0000","last_undegraded":"2024-09-17T21:48:55.755865+0000","last_fullsized":"2024-09-17T21:48:55.755865+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:24.237386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":63282450,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":38,"num_read":11272,"num_read_kb":53038,"num_write":520,"num_write_kb":16718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":45049091,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665348+0000","last_change":"2024-09-17T21:48:51.674108+0000","last_active":"2024-09-17T21:48:58.665348+0000","last_peered":"2024-09-17T21:48:58.665348+0000","last_clean":"2024-09-17T21:48:58.665348+0000","last_became_active":"2024-09-17T21:48:51.673749+0000","last_became_peered":"2024-09-17T21:48:51.673749+0000","last_unstale":"2024-09-17T21:48:58.665348+0000","last_undegraded":"2024-09-17T21:48:58.665348+0000","last_fullsized":"2024-09-17T21:48:58.665348+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:01.443679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"153'469","reported_seq":6137,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664719+0000","last_change":"2024-09-17T21:47:59.051638+0000","last_active":"2024-09-17T21:48:58.664719+0000","last_peered":"2024-09-17T21:48:58.664719+0000","last_clean":"2024-09-17T21:48:58.664719+0000","last_became_active":"2024-09-17T21:47:59.050777+0000","last_became_peered":"2024-09-17T21:47:59.050777+0000","last_unstale":"2024-09-17T21:48:58.664719+0000","last_undegraded":"2024-09-17T21:48:58.664719+0000","last_fullsized":"2024-09-17T21:48:58.664719+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:13.112037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":70024959,"num_objects":79,"num_object_clones":19,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":35,"num_read":6088,"num_read_kb":42149,"num_write":560,"num_write_kb":10380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":6854227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":261,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695044+0000","last_change":"2024-09-17T21:48:51.674836+0000","last_active":"2024-09-17T21:48:55.695044+0000","last_peered":"2024-09-17T21:48:55.695044+0000","last_clean":"2024-09-17T21:48:55.695044+0000","last_became_active":"2024-09-17T21:48:51.674445+0000","last_became_peered":"2024-09-17T21:48:51.674445+0000","last_unstale":"2024-09-17T21:48:55.695044+0000","last_undegraded":"2024-09-17T21:48:55.695044+0000","last_fullsized":"2024-09-17T21:48:55.695044+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:46.274757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"164'565","reported_seq":18308,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664669+0000","last_change":"2024-09-17T21:48:54.673341+0000","last_active":"2024-09-17T21:48:58.664669+0000","last_peered":"2024-09-17T21:48:58.664669+0000","last_clean":"2024-09-17T21:48:58.664669+0000","last_became_active":"2024-09-17T21:48:54.672047+0000","last_became_peered":"2024-09-17T21:48:54.672047+0000","last_unstale":"2024-09-17T21:48:58.664669+0000","last_undegraded":"2024-09-17T21:48:58.664669+0000","last_fullsized":"2024-09-17T21:48:58.664669+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:31.630366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":97054240,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":29,"num_read":18053,"num_read_kb":86870,"num_write":416,"num_write_kb":12074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":36726912,"num_keys_recovered":188,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664964+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:48:58.664964+0000","last_peered":"2024-09-17T21:48:58.664964+0000","last_clean":"2024-09-17T21:48:58.664964+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:48:58.664964+0000","last_undegraded":"2024-09-17T21:48:58.664964+0000","last_fullsized":"2024-09-17T21:48:58.664964+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"154'435","reported_seq":12202,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670413+0000","last_change":"2024-09-17T21:48:51.674368+0000","last_active":"2024-09-17T21:48:55.670413+0000","last_peered":"2024-09-17T21:48:55.670413+0000","last_clean":"2024-09-17T21:48:55.670413+0000","last_became_active":"2024-09-17T21:48:51.673390+0000","last_became_peered":"2024-09-17T21:48:51.673390+0000","last_unstale":"2024-09-17T21:48:55.670413+0000","last_undegraded":"2024-09-17T21:48:55.670413+0000","last_fullsized":"2024-09-17T21:48:55.670413+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:37:21.005654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":44775036,"num_objects":82,"num_object_clones":15,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":34,"num_read":11343,"num_read_kb":39369,"num_write":434,"num_write_kb":15631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":21875572,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"153'686","reported_seq":10624,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670048+0000","last_change":"2024-09-17T21:48:07.678657+0000","last_active":"2024-09-17T21:48:54.670048+0000","last_peered":"2024-09-17T21:48:54.670048+0000","last_clean":"2024-09-17T21:48:54.670048+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:48:54.670048+0000","last_undegraded":"2024-09-17T21:48:54.670048+0000","last_fullsized":"2024-09-17T21:48:54.670048+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:02.211561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":39181329,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":12942,"num_read_kb":37042,"num_write":935,"num_write_kb":8879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":330,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670235+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:48:55.670235+0000","last_peered":"2024-09-17T21:48:55.670235+0000","last_clean":"2024-09-17T21:48:55.670235+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:48:55.670235+0000","last_undegraded":"2024-09-17T21:48:55.670235+0000","last_fullsized":"2024-09-17T21:48:55.670235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'363","reported_seq":6247,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664590+0000","last_change":"2024-09-17T21:47:59.050146+0000","last_active":"2024-09-17T21:48:58.664590+0000","last_peered":"2024-09-17T21:48:58.664590+0000","last_clean":"2024-09-17T21:48:58.664590+0000","last_became_active":"2024-09-17T21:47:59.049352+0000","last_became_peered":"2024-09-17T21:47:59.049352+0000","last_unstale":"2024-09-17T21:48:58.664590+0000","last_undegraded":"2024-09-17T21:48:58.664590+0000","last_fullsized":"2024-09-17T21:48:58.664590+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:16:52.713465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":32596603,"num_objects":67,"num_object_clones":12,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":9019,"num_read_kb":28675,"num_write":296,"num_write_kb":5150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":1913979,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665084+0000","last_change":"2024-09-17T21:48:07.678232+0000","last_active":"2024-09-17T21:48:58.665084+0000","last_peered":"2024-09-17T21:48:58.665084+0000","last_clean":"2024-09-17T21:48:58.665084+0000","last_became_active":"2024-09-17T21:48:07.677909+0000","last_became_peered":"2024-09-17T21:48:07.677909+0000","last_unstale":"2024-09-17T21:48:58.665084+0000","last_undegraded":"2024-09-17T21:48:58.665084+0000","last_fullsized":"2024-09-17T21:48:58.665084+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:31.593828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"153'1724","reported_seq":8391,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695014+0000","last_change":"2024-09-17T21:48:10.691752+0000","last_active":"2024-09-17T21:48:55.695014+0000","last_peered":"2024-09-17T21:48:55.695014+0000","last_clean":"2024-09-17T21:48:55.695014+0000","last_became_active":"2024-09-17T21:48:10.690220+0000","last_became_peered":"2024-09-17T21:48:10.690220+0000","last_unstale":"2024-09-17T21:48:55.695014+0000","last_undegraded":"2024-09-17T21:48:55.695014+0000","last_fullsized":"2024-09-17T21:48:55.695014+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1724,"log_dups_size":0,"ondisk_log_size":1724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:35.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":47912313,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":14962,"num_read_kb":56309,"num_write":2635,"num_write_kb":12507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665447+0000","last_change":"2024-09-17T21:48:51.672488+0000","last_active":"2024-09-17T21:48:58.665447+0000","last_peered":"2024-09-17T21:48:58.665447+0000","last_clean":"2024-09-17T21:48:58.665447+0000","last_became_active":"2024-09-17T21:48:51.672125+0000","last_became_peered":"2024-09-17T21:48:51.672125+0000","last_unstale":"2024-09-17T21:48:58.665447+0000","last_undegraded":"2024-09-17T21:48:58.665447+0000","last_fullsized":"2024-09-17T21:48:58.665447+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:25:21.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"154'437","reported_seq":5240,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664931+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:48:58.664931+0000","last_peered":"2024-09-17T21:48:58.664931+0000","last_clean":"2024-09-17T21:48:58.664931+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:48:58.664931+0000","last_undegraded":"2024-09-17T21:48:58.664931+0000","last_fullsized":"2024-09-17T21:48:58.664931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:23:01.854917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":41601700,"num_objects":65,"num_object_clones":10,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":30,"num_read":5721,"num_read_kb":32054,"num_write":508,"num_write_kb":18446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":338,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670309+0000","last_change":"2024-09-17T21:47:56.035222+0000","last_active":"2024-09-17T21:48:55.670309+0000","last_peered":"2024-09-17T21:48:55.670309+0000","last_clean":"2024-09-17T21:48:55.670309+0000","last_became_active":"2024-09-17T21:47:56.032752+0000","last_became_peered":"2024-09-17T21:47:56.032752+0000","last_unstale":"2024-09-17T21:48:55.670309+0000","last_undegraded":"2024-09-17T21:48:55.670309+0000","last_fullsized":"2024-09-17T21:48:55.670309+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:00.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":474,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695189+0000","last_change":"2024-09-17T21:48:54.672873+0000","last_active":"2024-09-17T21:48:55.695189+0000","last_peered":"2024-09-17T21:48:55.695189+0000","last_clean":"2024-09-17T21:48:55.695189+0000","last_became_active":"2024-09-17T21:48:54.672495+0000","last_became_peered":"2024-09-17T21:48:54.672495+0000","last_unstale":"2024-09-17T21:48:55.695189+0000","last_undegraded":"2024-09-17T21:48:55.695189+0000","last_fullsized":"2024-09-17T21:48:55.695189+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:53.482926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"173'411","reported_seq":6807,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664485+0000","last_change":"2024-09-17T21:47:59.050979+0000","last_active":"2024-09-17T21:48:58.664485+0000","last_peered":"2024-09-17T21:48:58.664485+0000","last_clean":"2024-09-17T21:48:58.664485+0000","last_became_active":"2024-09-17T21:47:59.050160+0000","last_became_peered":"2024-09-17T21:47:59.050160+0000","last_unstale":"2024-09-17T21:48:58.664485+0000","last_undegraded":"2024-09-17T21:48:58.664485+0000","last_fullsized":"2024-09-17T21:48:58.664485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:34.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":46895225,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":7178,"num_read_kb":57742,"num_write":476,"num_write_kb":6044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":2650745,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"164'692","reported_seq":9822,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670480+0000","last_change":"2024-09-17T21:47:59.049023+0000","last_active":"2024-09-17T21:48:54.670480+0000","last_peered":"2024-09-17T21:48:54.670480+0000","last_clean":"2024-09-17T21:48:54.670480+0000","last_became_active":"2024-09-17T21:47:59.047989+0000","last_became_peered":"2024-09-17T21:47:59.047989+0000","last_unstale":"2024-09-17T21:48:54.670480+0000","last_undegraded":"2024-09-17T21:48:54.670480+0000","last_fullsized":"2024-09-17T21:48:54.670480+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:30.925211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":72455412,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":14363,"num_read_kb":67973,"num_write":674,"num_write_kb":25407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":51158343,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'438","reported_seq":4635,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.673505+0000","last_change":"2024-09-17T21:48:54.673505+0000","last_active":"2024-09-17T21:48:54.673505+0000","last_peered":"2024-09-17T21:48:54.673505+0000","last_clean":"2024-09-17T21:48:54.673505+0000","last_became_active":"2024-09-17T21:48:54.672093+0000","last_became_peered":"2024-09-17T21:48:54.672093+0000","last_unstale":"2024-09-17T21:48:54.673505+0000","last_undegraded":"2024-09-17T21:48:54.673505+0000","last_fullsized":"2024-09-17T21:48:54.673505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:23.781547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":12837641,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":4133,"num_read_kb":13183,"num_write":478,"num_write_kb":2530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":2433285,"num_keys_recovered":187,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"174'396","reported_seq":10159,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665332+0000","last_change":"2024-09-17T21:48:10.690933+0000","last_active":"2024-09-17T21:48:58.665332+0000","last_peered":"2024-09-17T21:48:58.665332+0000","last_clean":"2024-09-17T21:48:58.665332+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:48:58.665332+0000","last_undegraded":"2024-09-17T21:48:58.665332+0000","last_fullsized":"2024-09-17T21:48:58.665332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:34.456784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":60186794,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":31,"num_read":9902,"num_read_kb":57379,"num_write":350,"num_write_kb":14360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"173'482","reported_seq":18445,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.136855+0000","last_change":"2024-09-17T21:48:07.677450+0000","last_active":"2024-09-17T21:48:56.136855+0000","last_peered":"2024-09-17T21:48:56.136855+0000","last_clean":"2024-09-17T21:48:56.136855+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:48:56.136855+0000","last_undegraded":"2024-09-17T21:48:56.136855+0000","last_fullsized":"2024-09-17T21:48:56.136855+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:35.215362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":48316412,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":35,"num_read":17267,"num_read_kb":76249,"num_write":519,"num_write_kb":16844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'739","reported_seq":9465,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670174+0000","last_change":"2024-09-17T21:48:51.675249+0000","last_active":"2024-09-17T21:48:54.670174+0000","last_peered":"2024-09-17T21:48:54.670174+0000","last_clean":"2024-09-17T21:48:54.670174+0000","last_became_active":"2024-09-17T21:48:51.673534+0000","last_became_peered":"2024-09-17T21:48:51.673534+0000","last_unstale":"2024-09-17T21:48:54.670174+0000","last_undegraded":"2024-09-17T21:48:54.670174+0000","last_fullsized":"2024-09-17T21:48:54.670174+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:22.028264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":125315721,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":24,"num_read":11291,"num_read_kb":73850,"num_write":847,"num_write_kb":32039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":51394808,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"173'364","reported_seq":5185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.844195+0000","last_change":"2024-09-17T21:47:07.910393+0000","last_active":"2024-09-17T21:48:55.844195+0000","last_peered":"2024-09-17T21:48:55.844195+0000","last_clean":"2024-09-17T21:48:55.844195+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:48:55.844195+0000","last_undegraded":"2024-09-17T21:48:55.844195+0000","last_fullsized":"2024-09-17T21:48:55.844195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.482349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":44639046,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":33,"num_read":5078,"num_read_kb":27760,"num_write":546,"num_write_kb":25835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"155'885","reported_seq":16715,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670505+0000","last_change":"2024-09-17T21:48:54.675553+0000","last_active":"2024-09-17T21:48:55.670505+0000","last_peered":"2024-09-17T21:48:55.670505+0000","last_clean":"2024-09-17T21:48:55.670505+0000","last_became_active":"2024-09-17T21:48:54.674377+0000","last_became_peered":"2024-09-17T21:48:54.674377+0000","last_unstale":"2024-09-17T21:48:55.670505+0000","last_undegraded":"2024-09-17T21:48:55.670505+0000","last_fullsized":"2024-09-17T21:48:55.670505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:20:56.911576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":40848138,"num_objects":79,"num_object_clones":15,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":33,"num_read":18959,"num_read_kb":65942,"num_write":1247,"num_write_kb":28614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":67854713,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"164'513","reported_seq":6683,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.768034+0000","last_change":"2024-09-17T21:48:08.687002+0000","last_active":"2024-09-17T21:48:55.768034+0000","last_peered":"2024-09-17T21:48:55.768034+0000","last_clean":"2024-09-17T21:48:55.768034+0000","last_became_active":"2024-09-17T21:48:08.680620+0000","last_became_peered":"2024-09-17T21:48:08.680620+0000","last_unstale":"2024-09-17T21:48:55.768034+0000","last_undegraded":"2024-09-17T21:48:55.768034+0000","last_fullsized":"2024-09-17T21:48:55.768034+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:14:43.424736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":38661261,"num_objects":66,"num_object_clones":7,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":27,"num_read":8112,"num_read_kb":24131,"num_write":553,"num_write_kb":5414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":4987381,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"154'467","reported_seq":5920,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.676841+0000","last_change":"2024-09-17T21:48:54.676841+0000","last_active":"2024-09-17T21:48:54.676841+0000","last_peered":"2024-09-17T21:48:54.676841+0000","last_clean":"2024-09-17T21:48:54.676841+0000","last_became_active":"2024-09-17T21:48:54.675279+0000","last_became_peered":"2024-09-17T21:48:54.675279+0000","last_unstale":"2024-09-17T21:48:54.676841+0000","last_undegraded":"2024-09-17T21:48:54.676841+0000","last_fullsized":"2024-09-17T21:48:54.676841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:35:17.639973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":94502616,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":41,"num_read":4727,"num_read_kb":61191,"num_write":546,"num_write_kb":31936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":117899496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"173'306","reported_seq":11442,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.800520+0000","last_change":"2024-09-17T21:48:54.673288+0000","last_active":"2024-09-17T21:48:55.800520+0000","last_peered":"2024-09-17T21:48:55.800520+0000","last_clean":"2024-09-17T21:48:55.800520+0000","last_became_active":"2024-09-17T21:48:54.672211+0000","last_became_peered":"2024-09-17T21:48:54.672211+0000","last_unstale":"2024-09-17T21:48:55.800520+0000","last_undegraded":"2024-09-17T21:48:55.800520+0000","last_fullsized":"2024-09-17T21:48:55.800520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:13.873795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":82751052,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":9173,"num_read_kb":45760,"num_write":409,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":50789756,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'396","reported_seq":9208,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.112183+0000","last_change":"2024-09-17T21:48:54.673957+0000","last_active":"2024-09-17T21:48:56.112183+0000","last_peered":"2024-09-17T21:48:56.112183+0000","last_clean":"2024-09-17T21:48:56.112183+0000","last_became_active":"2024-09-17T21:48:54.672887+0000","last_became_peered":"2024-09-17T21:48:54.672887+0000","last_unstale":"2024-09-17T21:48:56.112183+0000","last_undegraded":"2024-09-17T21:48:56.112183+0000","last_fullsized":"2024-09-17T21:48:56.112183+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:06.513330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":71753111,"num_objects":76,"num_object_clones":16,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":35,"num_read":8209,"num_read_kb":34342,"num_write":523,"num_write_kb":12359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":16204837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2114222659,"num_objects":2377,"num_object_clones":468,"num_object_copies":4754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2377,"num_whiteouts":1030,"num_read":356834,"num_read_kb":1803367,"num_write":19681,"num_write_kb":583984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8178,"num_bytes_recovered":972479002,"num_keys_recovered":1680,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082204160,"data_stored":1525385724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":17056,"ondisk_log_size":17056,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607594,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309168,"kb_used_data":253748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94062672,"statfs":{"total":96636764160,"available":96320176128,"internally_reserved":0,"allocated":259837952,"data_stored":378294562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":635655159815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249756,"kb_used_data":200516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94122084,"statfs":{"total":96636764160,"available":96381014016,"internally_reserved":0,"allocated":205328384,"data_stored":277606841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342204,"kb_used_data":266592,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94029636,"statfs":{"total":96636764160,"available":96286347264,"internally_reserved":0,"allocated":272990208,"data_stored":371368151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210776,"kb_used_data":161536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94161064,"statfs":{"total":96636764160,"available":96420929536,"internally_reserved":0,"allocated":165412864,"data_stored":220626113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196198400,"data_stored":269685211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162705408,"data_stored":219150577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64610304,"data_stored":97321755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":129966080,"data_stored":191277738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":270942208,"data_stored":370508314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":257781760,"data_stored":377442129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:49:00.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:49:00.080 INFO:teuthology.orchestra.run.smithi165.stdout:5910 punch from 0x2f92400 to 0x2f97800, (0x5400 bytes) 2024-09-17T21:49:00.082 INFO:teuthology.orchestra.run.smithi165.stdout:5911 read 0xfbe000 thru 0xfc022e (0x222f bytes) 2024-09-17T21:49:00.082 INFO:teuthology.orchestra.run.smithi165.stdout:5913 write 0x9a0b600 thru 0x9a0de24 (0x2825 bytes) 2024-09-17T21:49:00.092 INFO:teuthology.orchestra.run.smithi165.stdout:5915 trunc from 0x9ad1a00 to 0x2106c00 2024-09-17T21:49:00.133 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.133 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:00.156 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:00.154+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:00.214 INFO:teuthology.orchestra.run.smithi165.stdout:5917 read 0x1317c00 thru 0x131b140 (0x3541 bytes) 2024-09-17T21:49:00.216 INFO:teuthology.orchestra.run.smithi165.stdout:5918 read 0x7a6400 thru 0x7b48e8 (0xe4e9 bytes) 2024-09-17T21:49:00.219 INFO:teuthology.orchestra.run.smithi165.stdout:5922 punch from 0x19b1600 to 0x19c1400, (0xfe00 bytes) 2024-09-17T21:49:00.225 INFO:teuthology.orchestra.run.smithi165.stdout:5923 trunc from 0x2106c00 to 0x6cf0e00 2024-09-17T21:49:00.239 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:00.237+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:00.254 INFO:teuthology.orchestra.run.smithi165.stdout:5926 write 0xe783600 thru 0xe7929a3 (0xf3a4 bytes) 2024-09-17T21:49:00.256 INFO:teuthology.orchestra.run.smithi165.stdout:5927 write 0x4cab800 thru 0x4cb6c13 (0xb414 bytes) 2024-09-17T21:49:00.259 INFO:teuthology.orchestra.run.smithi165.stdout:5928 write 0x2490400 thru 0x2495c64 (0x5865 bytes) 2024-09-17T21:49:00.264 INFO:teuthology.orchestra.run.smithi165.stdout:5930 trunc from 0xe792a00 to 0xa1f5200 2024-09-17T21:49:00.270 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.271 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.278 INFO:teuthology.orchestra.run.smithi165.stdout:5932 read 0x54a4e00 thru 0x54b162b (0xc82c bytes) 2024-09-17T21:49:00.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:5935 read 0x89e9000 thru 0x89e9088 (0x89 bytes) 2024-09-17T21:49:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:5937 punch from 0x2ca000 to 0x2cbc00, (0x1c00 bytes) 2024-09-17T21:49:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:5938 punch from 0xf6ae00 to 0xf6d000, (0x2200 bytes) 2024-09-17T21:49:00.281 INFO:teuthology.orchestra.run.smithi165.stdout:5940 punch from 0x6e51000 to 0x6e54600, (0x3600 bytes) 2024-09-17T21:49:00.282 INFO:teuthology.orchestra.run.smithi165.stdout:5941 punch from 0x4e83600 to 0x4e89600, (0x6000 bytes) 2024-09-17T21:49:00.295 INFO:teuthology.orchestra.run.smithi165.stdout:5947 write 0xccdba00 thru 0xcce7d52 (0xc353 bytes) 2024-09-17T21:49:00.297 INFO:teuthology.orchestra.run.smithi165.stdout:5948 read 0x8baca00 thru 0x8bba64e (0xdc4f bytes) 2024-09-17T21:49:00.297 INFO:teuthology.orchestra.run.smithi165.stdout:5952 write 0x2d7c600 thru 0x2d7ef78 (0x2979 bytes) 2024-09-17T21:49:00.300 INFO:teuthology.orchestra.run.smithi165.stdout:5953 write 0x475e800 thru 0x476d1d0 (0xe9d1 bytes) 2024-09-17T21:49:00.300 INFO:teuthology.orchestra.run.smithi165.stdout:5954 read 0x2a61e00 thru 0x2a62f64 (0x1165 bytes) 2024-09-17T21:49:00.300 INFO:teuthology.orchestra.run.smithi165.stdout:5955 read 0x9c92200 thru 0x9c9812e (0x5f2f bytes) 2024-09-17T21:49:00.300 INFO:teuthology.orchestra.run.smithi165.stdout:5957 read 0x83cb400 thru 0x83d4e86 (0x9a87 bytes) 2024-09-17T21:49:00.300 INFO:teuthology.orchestra.run.smithi165.stdout:5958 read 0xc53c800 thru 0xc549aca (0xd2cb bytes) 2024-09-17T21:49:00.311 INFO:teuthology.orchestra.run.smithi165.stdout:5962 trunc from 0xcce7e00 to 0x27bd400 2024-09-17T21:49:00.336 INFO:teuthology.orchestra.run.smithi165.stdout:5965 read 0x1a4e00 thru 0x1b07a3 (0xb9a4 bytes) 2024-09-17T21:49:00.339 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:00.337+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:00.355 INFO:teuthology.orchestra.run.smithi165.stdout:5966 write 0x780aa00 thru 0x7812d3a (0x833b bytes) 2024-09-17T21:49:00.363 INFO:teuthology.orchestra.run.smithi165.stdout:5968 trunc from 0x7812e00 to 0x282400 2024-09-17T21:49:00.398 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.398 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.407 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:00.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:00.434 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:49:00.447 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-17T21:49:00.132352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2114813046,"num_objects":2381,"num_object_clones":468,"num_object_copies":4762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2381,"num_whiteouts":1030,"num_read":356950,"num_read_kb":1803467,"num_write":19842,"num_write_kb":586402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8193,"num_bytes_recovered":976611578,"num_keys_recovered":1680,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17176,"ondisk_log_size":17176,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1111904,"kb_used_data":882392,"kb_used_omap":6,"kb_used_meta":223225,"kb_avail":376375456,"statfs":{"total":386547056640,"available":385408466944,"internally_reserved":0,"allocated":903569408,"data_stored":1247895667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6380,"internal_metadata":228583188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":90970702,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4317,"num_read_kb":38712,"num_write":480,"num_write_kb":52112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":495,"num_bytes_recovered":64066528,"num_keys_recovered":95,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000566"},"pg_stats":[{"pgid":"3.1f","version":"153'321","reported_seq":10779,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695583+0000","last_change":"2024-09-17T21:48:54.673693+0000","last_active":"2024-09-17T21:48:55.695583+0000","last_peered":"2024-09-17T21:48:55.695583+0000","last_clean":"2024-09-17T21:48:55.695583+0000","last_became_active":"2024-09-17T21:48:54.672690+0000","last_became_peered":"2024-09-17T21:48:54.672690+0000","last_unstale":"2024-09-17T21:48:55.695583+0000","last_undegraded":"2024-09-17T21:48:55.695583+0000","last_fullsized":"2024-09-17T21:48:55.695583+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:19.152600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056915999999999998,"stat_sum":{"num_bytes":60983108,"num_objects":81,"num_object_clones":12,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":34,"num_read":11050,"num_read_kb":70720,"num_write":309,"num_write_kb":11933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":50607226,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"164'487","reported_seq":19659,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665251+0000","last_change":"2024-09-17T21:48:54.675720+0000","last_active":"2024-09-17T21:48:58.665251+0000","last_peered":"2024-09-17T21:48:58.665251+0000","last_clean":"2024-09-17T21:48:58.665251+0000","last_became_active":"2024-09-17T21:48:54.674393+0000","last_became_peered":"2024-09-17T21:48:54.674393+0000","last_unstale":"2024-09-17T21:48:58.665251+0000","last_undegraded":"2024-09-17T21:48:58.665251+0000","last_fullsized":"2024-09-17T21:48:58.665251+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:02.807770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085558800000000001,"stat_sum":{"num_bytes":115875888,"num_objects":97,"num_object_clones":26,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":41,"num_read":18708,"num_read_kb":109022,"num_write":393,"num_write_kb":34487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":559,"num_bytes_recovered":109255344,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1d","version":"173'327","reported_seq":4196,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.127275+0000","last_change":"2024-09-17T21:47:10.889674+0000","last_active":"2024-09-17T21:48:56.127275+0000","last_peered":"2024-09-17T21:48:56.127275+0000","last_clean":"2024-09-17T21:48:56.127275+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:48:56.127275+0000","last_undegraded":"2024-09-17T21:48:56.127275+0000","last_fullsized":"2024-09-17T21:48:56.127275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:34.713443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068411399999999999,"stat_sum":{"num_bytes":80021685,"num_objects":70,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":35,"num_read":4018,"num_read_kb":38346,"num_write":603,"num_write_kb":31640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"164'482","reported_seq":17394,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670347+0000","last_change":"2024-09-17T21:47:11.890576+0000","last_active":"2024-09-17T21:48:55.670347+0000","last_peered":"2024-09-17T21:48:55.670347+0000","last_clean":"2024-09-17T21:48:55.670347+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:48:55.670347+0000","last_undegraded":"2024-09-17T21:48:55.670347+0000","last_fullsized":"2024-09-17T21:48:55.670347+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:59.849594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001440747,"stat_sum":{"num_bytes":88333502,"num_objects":88,"num_object_clones":22,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":31,"num_read":18554,"num_read_kb":85866,"num_write":533,"num_write_kb":39738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"164'410","reported_seq":16479,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.751148+0000","last_change":"2024-09-17T21:48:08.676407+0000","last_active":"2024-09-17T21:48:55.751148+0000","last_peered":"2024-09-17T21:48:55.751148+0000","last_clean":"2024-09-17T21:48:55.751148+0000","last_became_active":"2024-09-17T21:48:08.675352+0000","last_became_peered":"2024-09-17T21:48:08.675352+0000","last_unstale":"2024-09-17T21:48:55.751148+0000","last_undegraded":"2024-09-17T21:48:55.751148+0000","last_fullsized":"2024-09-17T21:48:55.751148+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":410,"log_dups_size":0,"ondisk_log_size":410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:05:15.599106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000514019,"stat_sum":{"num_bytes":70805726,"num_objects":67,"num_object_clones":16,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":25,"num_read":14331,"num_read_kb":42652,"num_write":341,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"164'574","reported_seq":10593,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670715+0000","last_change":"2024-09-17T21:47:10.889765+0000","last_active":"2024-09-17T21:48:54.670715+0000","last_peered":"2024-09-17T21:48:54.670715+0000","last_clean":"2024-09-17T21:48:54.670715+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:48:54.670715+0000","last_undegraded":"2024-09-17T21:48:54.670715+0000","last_fullsized":"2024-09-17T21:48:54.670715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:30:22.524109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095946699999999998,"stat_sum":{"num_bytes":101254943,"num_objects":74,"num_object_clones":15,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":31,"num_read":12545,"num_read_kb":80145,"num_write":627,"num_write_kb":25531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"164'364","reported_seq":4535,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664453+0000","last_change":"2024-09-17T21:47:11.890115+0000","last_active":"2024-09-17T21:48:58.664453+0000","last_peered":"2024-09-17T21:48:58.664453+0000","last_clean":"2024-09-17T21:48:58.664453+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:48:58.664453+0000","last_undegraded":"2024-09-17T21:48:58.664453+0000","last_fullsized":"2024-09-17T21:48:58.664453+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:25:01.508494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039334500000000002,"stat_sum":{"num_bytes":47057042,"num_objects":73,"num_object_clones":13,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":34,"num_read":4319,"num_read_kb":31678,"num_write":368,"num_write_kb":11312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"173'601","reported_seq":12627,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.116154+0000","last_change":"2024-09-17T21:48:08.686643+0000","last_active":"2024-09-17T21:48:56.116154+0000","last_peered":"2024-09-17T21:48:56.116154+0000","last_clean":"2024-09-17T21:48:56.116154+0000","last_became_active":"2024-09-17T21:48:08.681464+0000","last_became_peered":"2024-09-17T21:48:08.681464+0000","last_unstale":"2024-09-17T21:48:56.116154+0000","last_undegraded":"2024-09-17T21:48:56.116154+0000","last_fullsized":"2024-09-17T21:48:56.116154+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:59:57.602128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000485215,"stat_sum":{"num_bytes":90423561,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":11977,"num_read_kb":67143,"num_write":726,"num_write_kb":13790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":9646917,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"154'315","reported_seq":7169,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695481+0000","last_change":"2024-09-17T21:48:10.693039+0000","last_active":"2024-09-17T21:48:55.695481+0000","last_peered":"2024-09-17T21:48:55.695481+0000","last_clean":"2024-09-17T21:48:55.695481+0000","last_became_active":"2024-09-17T21:48:10.691486+0000","last_became_peered":"2024-09-17T21:48:10.691486+0000","last_unstale":"2024-09-17T21:48:55.695481+0000","last_undegraded":"2024-09-17T21:48:55.695481+0000","last_fullsized":"2024-09-17T21:48:55.695481+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:43.032485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059713300000000002,"stat_sum":{"num_bytes":19967504,"num_objects":61,"num_object_clones":11,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":34,"num_read":7364,"num_read_kb":30958,"num_write":302,"num_write_kb":8410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"170'866","reported_seq":10844,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.726454+0000","last_change":"2024-09-17T21:48:55.672954+0000","last_active":"2024-09-17T21:48:55.726454+0000","last_peered":"2024-09-17T21:48:55.726454+0000","last_clean":"2024-09-17T21:48:55.726454+0000","last_became_active":"2024-09-17T21:48:55.671674+0000","last_became_peered":"2024-09-17T21:48:55.671674+0000","last_unstale":"2024-09-17T21:48:55.726454+0000","last_undegraded":"2024-09-17T21:48:55.726454+0000","last_fullsized":"2024-09-17T21:48:55.726454+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:54.618156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077318899999999997,"stat_sum":{"num_bytes":63860296,"num_objects":90,"num_object_clones":20,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":47,"num_read":13380,"num_read_kb":73472,"num_write":946,"num_write_kb":17143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":453,"num_bytes_recovered":46911229,"num_keys_recovered":153,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"164'642","reported_seq":9385,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.748304+0000","last_change":"2024-09-17T21:48:51.685446+0000","last_active":"2024-09-17T21:48:55.748304+0000","last_peered":"2024-09-17T21:48:55.748304+0000","last_clean":"2024-09-17T21:48:55.748304+0000","last_became_active":"2024-09-17T21:48:51.675643+0000","last_became_peered":"2024-09-17T21:48:51.675643+0000","last_unstale":"2024-09-17T21:48:55.748304+0000","last_undegraded":"2024-09-17T21:48:55.748304+0000","last_fullsized":"2024-09-17T21:48:55.748304+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:38:21.493584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001198403,"stat_sum":{"num_bytes":53377608,"num_objects":76,"num_object_clones":17,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":32,"num_read":16335,"num_read_kb":58528,"num_write":637,"num_write_kb":14179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":33289947,"num_keys_recovered":140,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"170'333","reported_seq":17669,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670049+0000","last_change":"2024-09-17T21:47:17.916575+0000","last_active":"2024-09-17T21:48:54.670049+0000","last_peered":"2024-09-17T21:48:54.670049+0000","last_clean":"2024-09-17T21:48:54.670049+0000","last_became_active":"2024-09-17T21:47:17.915249+0000","last_became_peered":"2024-09-17T21:47:17.915249+0000","last_unstale":"2024-09-17T21:48:54.670049+0000","last_undegraded":"2024-09-17T21:48:54.670049+0000","last_fullsized":"2024-09-17T21:48:54.670049+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:02:21.536664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000761657,"stat_sum":{"num_bytes":146670737,"num_objects":81,"num_object_clones":19,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":28,"num_read":16461,"num_read_kb":133829,"num_write":424,"num_write_kb":41720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":39957736,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":276,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670305+0000","last_change":"2024-09-17T21:48:51.674495+0000","last_active":"2024-09-17T21:48:54.670305+0000","last_peered":"2024-09-17T21:48:54.670305+0000","last_clean":"2024-09-17T21:48:54.670305+0000","last_became_active":"2024-09-17T21:48:51.674180+0000","last_became_peered":"2024-09-17T21:48:51.674180+0000","last_unstale":"2024-09-17T21:48:54.670305+0000","last_undegraded":"2024-09-17T21:48:54.670305+0000","last_fullsized":"2024-09-17T21:48:54.670305+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:18:03.263647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"164'566","reported_seq":9672,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.755865+0000","last_change":"2024-09-17T21:48:10.693505+0000","last_active":"2024-09-17T21:48:55.755865+0000","last_peered":"2024-09-17T21:48:55.755865+0000","last_clean":"2024-09-17T21:48:55.755865+0000","last_became_active":"2024-09-17T21:48:10.691607+0000","last_became_peered":"2024-09-17T21:48:10.691607+0000","last_unstale":"2024-09-17T21:48:55.755865+0000","last_undegraded":"2024-09-17T21:48:55.755865+0000","last_fullsized":"2024-09-17T21:48:55.755865+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:41:24.237386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049646499999999997,"stat_sum":{"num_bytes":63282450,"num_objects":86,"num_object_clones":18,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":38,"num_read":11272,"num_read_kb":53038,"num_write":520,"num_write_kb":16718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":45049091,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":356,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665348+0000","last_change":"2024-09-17T21:48:51.674108+0000","last_active":"2024-09-17T21:48:58.665348+0000","last_peered":"2024-09-17T21:48:58.665348+0000","last_clean":"2024-09-17T21:48:58.665348+0000","last_became_active":"2024-09-17T21:48:51.673749+0000","last_became_peered":"2024-09-17T21:48:51.673749+0000","last_unstale":"2024-09-17T21:48:58.665348+0000","last_undegraded":"2024-09-17T21:48:58.665348+0000","last_fullsized":"2024-09-17T21:48:58.665348+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:02:01.443679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"153'469","reported_seq":6137,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664719+0000","last_change":"2024-09-17T21:47:59.051638+0000","last_active":"2024-09-17T21:48:58.664719+0000","last_peered":"2024-09-17T21:48:58.664719+0000","last_clean":"2024-09-17T21:48:58.664719+0000","last_became_active":"2024-09-17T21:47:59.050777+0000","last_became_peered":"2024-09-17T21:47:59.050777+0000","last_unstale":"2024-09-17T21:48:58.664719+0000","last_undegraded":"2024-09-17T21:48:58.664719+0000","last_fullsized":"2024-09-17T21:48:58.664719+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:13.112037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065823800000000001,"stat_sum":{"num_bytes":70024959,"num_objects":79,"num_object_clones":19,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":35,"num_read":6088,"num_read_kb":42149,"num_write":560,"num_write_kb":10380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":87,"num_bytes_recovered":6854227,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":261,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695044+0000","last_change":"2024-09-17T21:48:51.674836+0000","last_active":"2024-09-17T21:48:55.695044+0000","last_peered":"2024-09-17T21:48:55.695044+0000","last_clean":"2024-09-17T21:48:55.695044+0000","last_became_active":"2024-09-17T21:48:51.674445+0000","last_became_peered":"2024-09-17T21:48:51.674445+0000","last_unstale":"2024-09-17T21:48:55.695044+0000","last_undegraded":"2024-09-17T21:48:55.695044+0000","last_fullsized":"2024-09-17T21:48:55.695044+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:14:46.274757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"164'565","reported_seq":18308,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664669+0000","last_change":"2024-09-17T21:48:54.673341+0000","last_active":"2024-09-17T21:48:58.664669+0000","last_peered":"2024-09-17T21:48:58.664669+0000","last_clean":"2024-09-17T21:48:58.664669+0000","last_became_active":"2024-09-17T21:48:54.672047+0000","last_became_peered":"2024-09-17T21:48:54.672047+0000","last_unstale":"2024-09-17T21:48:58.664669+0000","last_undegraded":"2024-09-17T21:48:58.664669+0000","last_fullsized":"2024-09-17T21:48:58.664669+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:29:31.630366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068128100000000005,"stat_sum":{"num_bytes":97054240,"num_objects":70,"num_object_clones":14,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":29,"num_read":18053,"num_read_kb":86870,"num_write":416,"num_write_kb":12074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":455,"num_bytes_recovered":36726912,"num_keys_recovered":188,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":369,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664964+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:48:58.664964+0000","last_peered":"2024-09-17T21:48:58.664964+0000","last_clean":"2024-09-17T21:48:58.664964+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:48:58.664964+0000","last_undegraded":"2024-09-17T21:48:58.664964+0000","last_fullsized":"2024-09-17T21:48:58.664964+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"154'435","reported_seq":12202,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670413+0000","last_change":"2024-09-17T21:48:51.674368+0000","last_active":"2024-09-17T21:48:55.670413+0000","last_peered":"2024-09-17T21:48:55.670413+0000","last_clean":"2024-09-17T21:48:55.670413+0000","last_became_active":"2024-09-17T21:48:51.673390+0000","last_became_peered":"2024-09-17T21:48:51.673390+0000","last_unstale":"2024-09-17T21:48:55.670413+0000","last_undegraded":"2024-09-17T21:48:55.670413+0000","last_fullsized":"2024-09-17T21:48:55.670413+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":435,"log_dups_size":0,"ondisk_log_size":435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:37:21.005654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001233661,"stat_sum":{"num_bytes":44775036,"num_objects":82,"num_object_clones":15,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":34,"num_read":11343,"num_read_kb":39369,"num_write":434,"num_write_kb":15631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":21875572,"num_keys_recovered":36,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"153'686","reported_seq":10624,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670048+0000","last_change":"2024-09-17T21:48:07.678657+0000","last_active":"2024-09-17T21:48:54.670048+0000","last_peered":"2024-09-17T21:48:54.670048+0000","last_clean":"2024-09-17T21:48:54.670048+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:48:54.670048+0000","last_undegraded":"2024-09-17T21:48:54.670048+0000","last_fullsized":"2024-09-17T21:48:54.670048+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":686,"log_dups_size":0,"ondisk_log_size":686,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:21:02.211561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064635500000000004,"stat_sum":{"num_bytes":39181329,"num_objects":62,"num_object_clones":10,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":23,"num_read":12942,"num_read_kb":37042,"num_write":935,"num_write_kb":8879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":330,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670235+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:48:55.670235+0000","last_peered":"2024-09-17T21:48:55.670235+0000","last_clean":"2024-09-17T21:48:55.670235+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:48:55.670235+0000","last_undegraded":"2024-09-17T21:48:55.670235+0000","last_fullsized":"2024-09-17T21:48:55.670235+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"164'363","reported_seq":6247,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664590+0000","last_change":"2024-09-17T21:47:59.050146+0000","last_active":"2024-09-17T21:48:58.664590+0000","last_peered":"2024-09-17T21:48:58.664590+0000","last_clean":"2024-09-17T21:48:58.664590+0000","last_became_active":"2024-09-17T21:47:59.049352+0000","last_became_peered":"2024-09-17T21:47:59.049352+0000","last_unstale":"2024-09-17T21:48:58.664590+0000","last_undegraded":"2024-09-17T21:48:58.664590+0000","last_fullsized":"2024-09-17T21:48:58.664590+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:16:52.713465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054694399999999995,"stat_sum":{"num_bytes":32596603,"num_objects":67,"num_object_clones":12,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":32,"num_read":9019,"num_read_kb":28675,"num_write":296,"num_write_kb":5150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":1913979,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":137,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665084+0000","last_change":"2024-09-17T21:48:07.678232+0000","last_active":"2024-09-17T21:48:58.665084+0000","last_peered":"2024-09-17T21:48:58.665084+0000","last_clean":"2024-09-17T21:48:58.665084+0000","last_became_active":"2024-09-17T21:48:07.677909+0000","last_became_peered":"2024-09-17T21:48:07.677909+0000","last_unstale":"2024-09-17T21:48:58.665084+0000","last_undegraded":"2024-09-17T21:48:58.665084+0000","last_fullsized":"2024-09-17T21:48:58.665084+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:31.593828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"153'1724","reported_seq":8391,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695014+0000","last_change":"2024-09-17T21:48:10.691752+0000","last_active":"2024-09-17T21:48:55.695014+0000","last_peered":"2024-09-17T21:48:55.695014+0000","last_clean":"2024-09-17T21:48:55.695014+0000","last_became_active":"2024-09-17T21:48:10.690220+0000","last_became_peered":"2024-09-17T21:48:10.690220+0000","last_unstale":"2024-09-17T21:48:55.695014+0000","last_undegraded":"2024-09-17T21:48:55.695014+0000","last_fullsized":"2024-09-17T21:48:55.695014+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1724,"log_dups_size":0,"ondisk_log_size":1724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:53:35.986440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063509600000000003,"stat_sum":{"num_bytes":47912313,"num_objects":76,"num_object_clones":13,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":36,"num_read":14962,"num_read_kb":56309,"num_write":2635,"num_write_kb":12507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":148,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665447+0000","last_change":"2024-09-17T21:48:51.672488+0000","last_active":"2024-09-17T21:48:58.665447+0000","last_peered":"2024-09-17T21:48:58.665447+0000","last_clean":"2024-09-17T21:48:58.665447+0000","last_became_active":"2024-09-17T21:48:51.672125+0000","last_became_peered":"2024-09-17T21:48:51.672125+0000","last_unstale":"2024-09-17T21:48:58.665447+0000","last_undegraded":"2024-09-17T21:48:58.665447+0000","last_fullsized":"2024-09-17T21:48:58.665447+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:25:21.180009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"154'437","reported_seq":5240,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664931+0000","last_change":"2024-09-17T21:46:03.279142+0000","last_active":"2024-09-17T21:48:58.664931+0000","last_peered":"2024-09-17T21:48:58.664931+0000","last_clean":"2024-09-17T21:48:58.664931+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:48:58.664931+0000","last_undegraded":"2024-09-17T21:48:58.664931+0000","last_fullsized":"2024-09-17T21:48:58.664931+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:23:01.854917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069204200000000003,"stat_sum":{"num_bytes":41601700,"num_objects":65,"num_object_clones":10,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":30,"num_read":5721,"num_read_kb":32054,"num_write":508,"num_write_kb":18446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":338,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670309+0000","last_change":"2024-09-17T21:47:56.035222+0000","last_active":"2024-09-17T21:48:55.670309+0000","last_peered":"2024-09-17T21:48:55.670309+0000","last_clean":"2024-09-17T21:48:55.670309+0000","last_became_active":"2024-09-17T21:47:56.032752+0000","last_became_peered":"2024-09-17T21:47:56.032752+0000","last_unstale":"2024-09-17T21:48:55.670309+0000","last_undegraded":"2024-09-17T21:48:55.670309+0000","last_fullsized":"2024-09-17T21:48:55.670309+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:55:00.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":474,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.695189+0000","last_change":"2024-09-17T21:48:54.672873+0000","last_active":"2024-09-17T21:48:55.695189+0000","last_peered":"2024-09-17T21:48:55.695189+0000","last_clean":"2024-09-17T21:48:55.695189+0000","last_became_active":"2024-09-17T21:48:54.672495+0000","last_became_peered":"2024-09-17T21:48:54.672495+0000","last_unstale":"2024-09-17T21:48:55.695189+0000","last_undegraded":"2024-09-17T21:48:55.695189+0000","last_fullsized":"2024-09-17T21:48:55.695189+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:53.482926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"173'411","reported_seq":6807,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.664485+0000","last_change":"2024-09-17T21:47:59.050979+0000","last_active":"2024-09-17T21:48:58.664485+0000","last_peered":"2024-09-17T21:48:58.664485+0000","last_clean":"2024-09-17T21:48:58.664485+0000","last_became_active":"2024-09-17T21:47:59.050160+0000","last_became_peered":"2024-09-17T21:47:59.050160+0000","last_unstale":"2024-09-17T21:48:58.664485+0000","last_undegraded":"2024-09-17T21:48:58.664485+0000","last_fullsized":"2024-09-17T21:48:58.664485+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:34.277963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094565199999999995,"stat_sum":{"num_bytes":46895225,"num_objects":64,"num_object_clones":10,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":30,"num_read":7178,"num_read_kb":57742,"num_write":476,"num_write_kb":6044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":2650745,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"164'692","reported_seq":9822,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670480+0000","last_change":"2024-09-17T21:47:59.049023+0000","last_active":"2024-09-17T21:48:54.670480+0000","last_peered":"2024-09-17T21:48:54.670480+0000","last_clean":"2024-09-17T21:48:54.670480+0000","last_became_active":"2024-09-17T21:47:59.047989+0000","last_became_peered":"2024-09-17T21:47:59.047989+0000","last_unstale":"2024-09-17T21:48:54.670480+0000","last_undegraded":"2024-09-17T21:48:54.670480+0000","last_fullsized":"2024-09-17T21:48:54.670480+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:33:30.925211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000331157,"stat_sum":{"num_bytes":72455412,"num_objects":66,"num_object_clones":14,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":24,"num_read":14363,"num_read_kb":67973,"num_write":674,"num_write_kb":25407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":51158343,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"164'438","reported_seq":4635,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.673505+0000","last_change":"2024-09-17T21:48:54.673505+0000","last_active":"2024-09-17T21:48:54.673505+0000","last_peered":"2024-09-17T21:48:54.673505+0000","last_clean":"2024-09-17T21:48:54.673505+0000","last_became_active":"2024-09-17T21:48:54.672093+0000","last_became_peered":"2024-09-17T21:48:54.672093+0000","last_unstale":"2024-09-17T21:48:54.673505+0000","last_undegraded":"2024-09-17T21:48:54.673505+0000","last_fullsized":"2024-09-17T21:48:54.673505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:23.781547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011217499999999999,"stat_sum":{"num_bytes":12837641,"num_objects":54,"num_object_clones":7,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":25,"num_read":4133,"num_read_kb":13183,"num_write":478,"num_write_kb":2530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":2433285,"num_keys_recovered":187,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"174'396","reported_seq":10159,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-17T21:48:58.665332+0000","last_change":"2024-09-17T21:48:10.690933+0000","last_active":"2024-09-17T21:48:58.665332+0000","last_peered":"2024-09-17T21:48:58.665332+0000","last_clean":"2024-09-17T21:48:58.665332+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:48:58.665332+0000","last_undegraded":"2024-09-17T21:48:58.665332+0000","last_fullsized":"2024-09-17T21:48:58.665332+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:11:34.456784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045591000000000002,"stat_sum":{"num_bytes":60186794,"num_objects":83,"num_object_clones":14,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":31,"num_read":9902,"num_read_kb":57379,"num_write":350,"num_write_kb":14360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"173'482","reported_seq":18445,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.136855+0000","last_change":"2024-09-17T21:48:07.677450+0000","last_active":"2024-09-17T21:48:56.136855+0000","last_peered":"2024-09-17T21:48:56.136855+0000","last_clean":"2024-09-17T21:48:56.136855+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:48:56.136855+0000","last_undegraded":"2024-09-17T21:48:56.136855+0000","last_fullsized":"2024-09-17T21:48:56.136855+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:18:35.215362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056174199999999995,"stat_sum":{"num_bytes":48316412,"num_objects":77,"num_object_clones":16,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":35,"num_read":17267,"num_read_kb":76249,"num_write":519,"num_write_kb":16844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"161'739","reported_seq":9465,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.670174+0000","last_change":"2024-09-17T21:48:51.675249+0000","last_active":"2024-09-17T21:48:54.670174+0000","last_peered":"2024-09-17T21:48:54.670174+0000","last_clean":"2024-09-17T21:48:54.670174+0000","last_became_active":"2024-09-17T21:48:51.673534+0000","last_became_peered":"2024-09-17T21:48:51.673534+0000","last_unstale":"2024-09-17T21:48:54.670174+0000","last_undegraded":"2024-09-17T21:48:54.670174+0000","last_fullsized":"2024-09-17T21:48:54.670174+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":739,"log_dups_size":0,"ondisk_log_size":739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:02:22.028264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011860740000000001,"stat_sum":{"num_bytes":125315721,"num_objects":68,"num_object_clones":16,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":24,"num_read":11291,"num_read_kb":73850,"num_write":847,"num_write_kb":32039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":51394808,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"173'364","reported_seq":5185,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.844195+0000","last_change":"2024-09-17T21:47:07.910393+0000","last_active":"2024-09-17T21:48:55.844195+0000","last_peered":"2024-09-17T21:48:55.844195+0000","last_clean":"2024-09-17T21:48:55.844195+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:48:55.844195+0000","last_undegraded":"2024-09-17T21:48:55.844195+0000","last_fullsized":"2024-09-17T21:48:55.844195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":364,"log_dups_size":0,"ondisk_log_size":364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:42:32.482349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014475309999999999,"stat_sum":{"num_bytes":44639046,"num_objects":68,"num_object_clones":11,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":33,"num_read":5078,"num_read_kb":27760,"num_write":546,"num_write_kb":25835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"155'885","reported_seq":16715,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.670505+0000","last_change":"2024-09-17T21:48:54.675553+0000","last_active":"2024-09-17T21:48:55.670505+0000","last_peered":"2024-09-17T21:48:55.670505+0000","last_clean":"2024-09-17T21:48:55.670505+0000","last_became_active":"2024-09-17T21:48:54.674377+0000","last_became_peered":"2024-09-17T21:48:54.674377+0000","last_unstale":"2024-09-17T21:48:55.670505+0000","last_undegraded":"2024-09-17T21:48:55.670505+0000","last_fullsized":"2024-09-17T21:48:55.670505+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:20:56.911576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011364789999999999,"stat_sum":{"num_bytes":40848138,"num_objects":79,"num_object_clones":15,"num_object_copies":158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":33,"num_read":18959,"num_read_kb":65942,"num_write":1247,"num_write_kb":28614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":67854713,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"164'513","reported_seq":6683,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.768034+0000","last_change":"2024-09-17T21:48:08.687002+0000","last_active":"2024-09-17T21:48:55.768034+0000","last_peered":"2024-09-17T21:48:55.768034+0000","last_clean":"2024-09-17T21:48:55.768034+0000","last_became_active":"2024-09-17T21:48:08.680620+0000","last_became_peered":"2024-09-17T21:48:08.680620+0000","last_unstale":"2024-09-17T21:48:55.768034+0000","last_undegraded":"2024-09-17T21:48:55.768034+0000","last_fullsized":"2024-09-17T21:48:55.768034+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:14:43.424736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056784399999999997,"stat_sum":{"num_bytes":38661261,"num_objects":66,"num_object_clones":7,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":27,"num_read":8112,"num_read_kb":24131,"num_write":553,"num_write_kb":5414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":4987381,"num_keys_recovered":22,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"154'467","reported_seq":5920,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-17T21:48:54.676841+0000","last_change":"2024-09-17T21:48:54.676841+0000","last_active":"2024-09-17T21:48:54.676841+0000","last_peered":"2024-09-17T21:48:54.676841+0000","last_clean":"2024-09-17T21:48:54.676841+0000","last_became_active":"2024-09-17T21:48:54.675279+0000","last_became_peered":"2024-09-17T21:48:54.675279+0000","last_unstale":"2024-09-17T21:48:54.676841+0000","last_undegraded":"2024-09-17T21:48:54.676841+0000","last_fullsized":"2024-09-17T21:48:54.676841+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:35:17.639973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075546999999999995,"stat_sum":{"num_bytes":94502616,"num_objects":92,"num_object_clones":25,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":41,"num_read":4727,"num_read_kb":61191,"num_write":546,"num_write_kb":31936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":501,"num_bytes_recovered":117899496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"173'306","reported_seq":11442,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:55.800520+0000","last_change":"2024-09-17T21:48:54.673288+0000","last_active":"2024-09-17T21:48:55.800520+0000","last_peered":"2024-09-17T21:48:55.800520+0000","last_clean":"2024-09-17T21:48:55.800520+0000","last_became_active":"2024-09-17T21:48:54.672211+0000","last_became_peered":"2024-09-17T21:48:54.672211+0000","last_unstale":"2024-09-17T21:48:55.800520+0000","last_undegraded":"2024-09-17T21:48:55.800520+0000","last_fullsized":"2024-09-17T21:48:55.800520+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:13.873795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000420973,"stat_sum":{"num_bytes":82751052,"num_objects":63,"num_object_clones":10,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":31,"num_read":9173,"num_read_kb":45760,"num_write":409,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":50789756,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"173'396","reported_seq":9208,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-17T21:48:56.112183+0000","last_change":"2024-09-17T21:48:54.673957+0000","last_active":"2024-09-17T21:48:56.112183+0000","last_peered":"2024-09-17T21:48:56.112183+0000","last_clean":"2024-09-17T21:48:56.112183+0000","last_became_active":"2024-09-17T21:48:54.672887+0000","last_became_peered":"2024-09-17T21:48:54.672887+0000","last_unstale":"2024-09-17T21:48:56.112183+0000","last_undegraded":"2024-09-17T21:48:56.112183+0000","last_fullsized":"2024-09-17T21:48:56.112183+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":396,"log_dups_size":0,"ondisk_log_size":396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:06.513330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001135886,"stat_sum":{"num_bytes":71753111,"num_objects":76,"num_object_clones":16,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":35,"num_read":8209,"num_read_kb":34342,"num_write":523,"num_write_kb":12359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":16204837,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2114222659,"num_objects":2377,"num_object_clones":468,"num_object_copies":4754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2377,"num_whiteouts":1030,"num_read":356834,"num_read_kb":1803367,"num_write":19681,"num_write_kb":583984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8178,"num_bytes_recovered":972479002,"num_keys_recovered":1680,"num_objects_omap":57,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082204160,"data_stored":1525385724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":17056,"ondisk_log_size":17056,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":4132576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607594,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309168,"kb_used_data":253748,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94062672,"statfs":{"total":96636764160,"available":96320176128,"internally_reserved":0,"allocated":259837952,"data_stored":378294562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":0,"seq":635655159815,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":51539607593,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249756,"kb_used_data":200516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94122084,"statfs":{"total":96636764160,"available":96381014016,"internally_reserved":0,"allocated":205328384,"data_stored":277606841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342204,"kb_used_data":266592,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94029636,"statfs":{"total":96636764160,"available":96286347264,"internally_reserved":0,"allocated":272990208,"data_stored":371368151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210776,"kb_used_data":161536,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94161064,"statfs":{"total":96636764160,"available":96420929536,"internally_reserved":0,"allocated":165412864,"data_stored":220626113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":196198400,"data_stored":269685211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":162705408,"data_stored":219150577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64610304,"data_stored":97321755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":129966080,"data_stored":191277738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":270942208,"data_stored":370508314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":257781760,"data_stored":377442129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:49:00.449 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:49:00.450 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:00.450 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:49:00.450 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 1.000000 2024-09-17T21:49:00.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T21:49:00.451 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:00.449+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:00.460 INFO:teuthology.orchestra.run.smithi165.stdout:5970 write 0x7f8d000 thru 0x7f9aac6 (0xdac7 bytes) 2024-09-17T21:49:00.478 INFO:teuthology.orchestra.run.smithi165.stdout:5972 write 0xb014c00 thru 0xb019221 (0x4622 bytes) 2024-09-17T21:49:00.480 INFO:teuthology.orchestra.run.smithi165.stdout:5973 punch from 0x299ee00 to 0x29ad000, (0xe200 bytes) 2024-09-17T21:49:00.482 INFO:teuthology.orchestra.run.smithi165.stdout:5975 read 0x1d51400 thru 0x1d60b5e (0xf75f bytes) 2024-09-17T21:49:00.483 INFO:teuthology.orchestra.run.smithi165.stdout:5976 read 0x3955a00 thru 0x3956d71 (0x1372 bytes) 2024-09-17T21:49:00.483 INFO:teuthology.orchestra.run.smithi165.stdout:5977 read 0x3221400 thru 0x322af71 (0x9b72 bytes) 2024-09-17T21:49:00.484 INFO:teuthology.orchestra.run.smithi165.stdout:5978 read 0x4b16200 thru 0x4b16be2 (0x9e3 bytes) 2024-09-17T21:49:00.490 INFO:teuthology.orchestra.run.smithi165.stdout:5979 trunc from 0xb019400 to 0x67c7e00 2024-09-17T21:49:00.507 INFO:teuthology.orchestra.run.smithi165.stdout:5980 write 0x381ec00 thru 0x38259c7 (0x6dc8 bytes) 2024-09-17T21:49:00.521 INFO:teuthology.orchestra.run.smithi165.stdout:5982 write 0x7ee1400 thru 0x7eec4c1 (0xb0c2 bytes) 2024-09-17T21:49:00.522 INFO:teuthology.orchestra.run.smithi165.stdout:5983 write 0x3225600 thru 0x32302be (0xacbf bytes) 2024-09-17T21:49:00.523 INFO:teuthology.orchestra.run.smithi165.stdout:5984 punch from 0x4461e00 to 0x4471200, (0xf400 bytes) 2024-09-17T21:49:00.524 INFO:teuthology.orchestra.run.smithi165.stdout:5985 write 0x2d04000 thru 0x2d0ca35 (0x8a36 bytes) 2024-09-17T21:49:00.526 INFO:teuthology.orchestra.run.smithi165.stdout:5986 write 0x331d800 thru 0x3326d03 (0x9504 bytes) 2024-09-17T21:49:00.526 INFO:teuthology.orchestra.run.smithi165.stdout:5987 punch from 0x40ffc00 to 0x4103000, (0x3400 bytes) 2024-09-17T21:49:00.526 INFO:teuthology.orchestra.run.smithi165.stdout:5989 read 0x280e200 thru 0x2814e15 (0x6c16 bytes) 2024-09-17T21:49:00.532 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.532 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.541 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:00.538+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:00.543 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:00.544 INFO:teuthology.orchestra.run.smithi165.stdout:5990 write 0xc97c800 thru 0xc987505 (0xad06 bytes) 2024-09-17T21:49:00.544 INFO:teuthology.orchestra.run.smithi165.stdout:5991 punch from 0xc2d6800 to 0xc2d9a00, (0x3200 bytes) 2024-09-17T21:49:00.548 INFO:teuthology.orchestra.run.smithi165.stdout:5992 punch from 0x7af0600 to 0x7af8e00, (0x8800 bytes) 2024-09-17T21:49:00.553 INFO:teuthology.orchestra.run.smithi165.stdout:5993 trunc from 0xc987600 to 0x68ed800 2024-09-17T21:49:00.570 INFO:teuthology.orchestra.run.smithi165.stdout:5994 read 0x2db2a00 thru 0x2dbfae4 (0xd0e5 bytes) 2024-09-17T21:49:00.572 INFO:teuthology.orchestra.run.smithi165.stdout:5996 punch from 0x630d200 to 0x6319600, (0xc400 bytes) 2024-09-17T21:49:00.572 INFO:teuthology.orchestra.run.smithi165.stdout:5999 punch from 0x61cd800 to 0x61d2000, (0x4800 bytes) 2024-09-17T21:49:00.642 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:00.639+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:00.691 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.692 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:00.725 INFO:teuthology.orchestra.run.smithi165.stdout:checking clone #54, image image_client.0-clone54 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent55 2024-09-17T21:49:00.744 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:00.741+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:00.844 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:00.842+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:00.851 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.851 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T21:49:00.877 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:49:00.946 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:00.943+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:00.972 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:00.972 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:00.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:01.046 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:01.044+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:01.129 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:01.129 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:01.137 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:01.147 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:01.145+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:01.244 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:01.244 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:01.247 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:01.245+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:01.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:01.348 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:01.346+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:01.401 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:01.401 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:01.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:01.450 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:01.448+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:01.550 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:01.548+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:01.556 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:01.556 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:01.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:01.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:01.651 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:01.649+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:01.673 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:01.674 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:01.683 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:01.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:01.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:01.751 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:01.751 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:01.752 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:01.749+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:01.767 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:01.768 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:01.785 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:01.785 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:01.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:01.793 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:01.793 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:01.793 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:01.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:01.819 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:01.820 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:01.872 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:01.870+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:01.912 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:01.912 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:01.913 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:01.913 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:01.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:01.953 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:01.951+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:02.054 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:02.052+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:02.073 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:02.073 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 147.197398198, 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:02.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:02.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:02.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160951888, 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:02.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:02.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160930586999996, 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:02.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160909825999994, 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:02.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160888720000003, 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:02.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:02.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:02.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160868230999995, 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:02.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:02.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160844134000001, 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:02.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160821640999998, 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:02.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:02.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160794034000006, 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:02.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:02.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 75.160769193999997, 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:02.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.008815419000001, 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:02.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.008791657000003, 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:02.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.008749268000003, 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:02.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:02.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.953000614999993, 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:02.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952977520000005, 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:02.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:02.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952951139999996, 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:02.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952934503999998, 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:02.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:02.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952904540000006, 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:02.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952891614999999, 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:02.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:02.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 70.952881394000002, 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:02.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:02.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:02.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:02.172 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:02.169+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:02.273 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:02.271+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:02.289 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:02.289 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:02.298 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:02.373 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:02.371+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:02.406 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:02.406 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:02.406 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:02.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:02.498 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:02.496+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:02.563 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 156.821150109, 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:02.564 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:02.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 156.820105649, 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:02.570 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:02.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.618920317999994, 2024-09-17T21:49:02.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:02.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:02.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.403002650000005, 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:02.582 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:02.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.200565908000002, 2024-09-17T21:49:02.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:02.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:02.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.200508728000003, 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:02.594 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:02.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:02.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:02.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199938146999997, 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:02.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:02.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199910212999995, 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:02.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199775861000006, 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:02.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199754326000004, 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:02.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199593360999998, 2024-09-17T21:49:02.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:02.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:02.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.199537595999999, 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:02.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.198190988999997, 2024-09-17T21:49:02.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:02.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:02.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.198139432999994, 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.640 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:02.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:02.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.197451172000001, 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:02.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:02.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.197434121000001, 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:02.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:02.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.196495881000004, 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:02.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.196465540999995, 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:02.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:02.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.902068092999997, 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:02.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:02.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:02.673 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.000950392999997, 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:02.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:02.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:02.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:02.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.678 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:02.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:02.679 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:02.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:02.681 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:02.678+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:02.781 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:02.778+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:02.828 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:02.828 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:02.828 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:02.836 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:02.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:49:02.920 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:02.918+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:02.982 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:02.980+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:02.999 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:03.000 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:03.000 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:03.000 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:03.000 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:03.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:03.083 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:03.081+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18781004600001, 2024-09-17T21:49:03.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:03.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18778578600001, 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:03.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.187759903, 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18773490199999, 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:03.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:03.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18769510300001, 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:03.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:03.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.187679358, 2024-09-17T21:49:03.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:03.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:03.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.187667823, 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18765752300001, 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:03.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18568628, 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:03.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18567313200001, 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:03.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18566252599999, 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:03.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18565218800001, 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:03.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:03.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18564218099999, 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:03.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:03.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18563223000001, 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:03.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:03.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18561692200001, 2024-09-17T21:49:03.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:03.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:03.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18363007900001, 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:03.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:03.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18361759000001, 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:03.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 166.18360712699999, 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:03.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.484271329, 2024-09-17T21:49:03.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:03.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.1295413529999996, 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:03.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:03.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.258 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:03.260 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:03.260 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:03.257+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:03.360 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:03.359+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:03.367 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:03.367 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:03.367 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:03.367 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:03.376 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:03.461 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:03.459+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:03.482 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:03.482 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:03.482 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:03.482 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:03.482 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:03.491 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:03.574 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:03.572+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:03.639 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663984491000001, 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:03.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663960354, 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:03.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663928099, 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:03.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:03.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663898046, 2024-09-17T21:49:03.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:03.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:03.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663874700000001, 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:03.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663821753000001, 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:03.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.663788406999998, 2024-09-17T21:49:03.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:03.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:03.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660457174000001, 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:03.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660429273999998, 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:03.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:03.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:03.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660406567999999, 2024-09-17T21:49:03.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:03.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.6603903, 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:03.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660380328999999, 2024-09-17T21:49:03.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:03.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.694 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660368179999999, 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:03.699 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660358485, 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:03.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660348892000002, 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.704 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.708 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660339401999998, 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:03.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660329556000001, 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:03.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:03.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.660320215999999, 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:03.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.962106801999999, 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:03.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:03.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.961516830000001, 2024-09-17T21:49:03.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:03.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:03.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:03.733 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:03.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:03.735 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:03.732+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:03.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:03.835 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:03.833+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:03.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:49:03.951 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:03.949+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:04.009 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.009 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.009 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:04.009 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:04.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:04.028 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:04.028 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:04.048 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:04.048 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:04.048 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:04.046+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:04.065 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:04.066 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:04.137 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:04.135+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:04.138 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.139 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:04.238 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:04.236+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:04.255 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.255 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:04.339 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:04.337+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:04.412 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.412 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:04.464 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:04.461+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:04.527 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.528 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.536 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:04.577 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:04.574+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:04.641 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:04.638+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:04.684 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.685 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:04.777 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:04.774+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:04.840 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:04.842 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:04.839+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:04.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:04.944 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:04.940+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:05.004 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.004 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:05.057 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:05.055+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:05.127 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.128 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:05.145 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:05.142+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:05.245 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:05.243+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:05.283 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.284 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.292 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:05.376 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:05.374+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:05.440 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.440 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.447 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:05.444+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:05.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:05.547 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:05.545+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:05.596 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.596 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:05.649 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:05.646+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:05.749 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:05.746+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:05.753 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.753 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:05.850 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:05.848+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:05.894 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0] out_osds: [2, 3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:05.894 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:05.894 INFO:tasks.thrashosds.thrasher:Adding osd 2 2024-09-17T21:49:05.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-17T21:49:05.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:49:05.915 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:05.915 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:05.924 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:05.982 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:05.979+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:06.051 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:06.048+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:06.077 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:06.077 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:06.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:06.085 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:06.152 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:06.149+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:06.194 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:06.194 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:06.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:06.205 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:06.205 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:06.223 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:06.237 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:49:06.234+0000 7f0c28614640 -1 mon.a@0(leader).osd e183 definitely_dead 0 2024-09-17T21:49:06.240 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:06.253 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:06.250+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:06.258 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:06.258 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:06.275 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:06.275 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:06.292 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:06.293 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:06.313 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:06.313 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338421008000001, 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:06.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:06.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338396871, 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:06.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338364616, 2024-09-17T21:49:06.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:06.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:06.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338334563, 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:06.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:06.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338311217000001, 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:06.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.337 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338258270000001, 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:06.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.338224923999999, 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.343 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:06.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334893691000001, 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:06.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334865790999999, 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.352 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:06.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334843084999999, 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:06.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:06.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334826817, 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:06.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334816845999999, 2024-09-17T21:49:06.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:06.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:06.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334804696999999, 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:06.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.375 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334795002, 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.376 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:06.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.380 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334785408999998, 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:06.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:06.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:06.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334775918999998, 2024-09-17T21:49:06.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:06.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:06.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:06.393 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334766073000001, 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:06.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:06.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.334756732999999, 2024-09-17T21:49:06.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:06.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.636543319000001, 2024-09-17T21:49:06.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.635953347000001, 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:06.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:06.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:06.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:06.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:06.414 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:06.416 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:06.413+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:06.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:06.517 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:06.515+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:06.524 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:06.524 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:06.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:06.617 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:06.614+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:06.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:06.642 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:06.642 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:06.642 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:06.642 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:06.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:06.735 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:06.732+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:06.801 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:06.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 165.11499616500001, 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:06.802 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:06.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 165.08272119, 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:06.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:06.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 165.082482422, 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.814 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:06.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.91497629200001, 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.819 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:06.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:06.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.908074521, 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.825 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:06.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:06.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.90795842999999, 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:06.832 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.837 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710954909999998, 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.838 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:06.839 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.840 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:06.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710916115000003, 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:06.844 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.845 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:06.847 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.848 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710901831000001, 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:06.849 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.851 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:06.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:06.853 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:06.854 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710656865000004, 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.855 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:06.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:06.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710574417000004, 2024-09-17T21:49:06.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:06.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.862 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:06.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.710542606999994, 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.866 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.323879491, 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:06.872 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.873 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:06.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:06.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 77.313950511000002, 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:06.878 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:06.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:06.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:06.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:06.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.121615796999997, 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:06.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:06.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:06.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.121358376000003, 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:06.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:06.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.12132553, 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:06.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:06.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:06.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.121308585000001, 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.908 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:06.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:06.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.121267088000003, 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:06.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:06.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:06.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 13.127030585, 2024-09-17T21:49:06.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:06.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:06.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:06.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:06.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:06.927 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:06.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:06.929 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.2. 2024-09-17T21:49:06.930 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:49:06.931 INFO:tasks.thrashosds.thrasher:Added osd 2 2024-09-17T21:49:06.931 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:06.927+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:07.029 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:07.026+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:28874 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:49:06.898626+0000", 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.177512427, 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.17753233400000001, 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:49:07.078 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 28874 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898626+0000", 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898626+0000", 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898627+0000", 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5570000000000001e-06 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898636+0000", 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3100000000000001e-06 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898655+0000", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8943000000000001e-05 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898666+0000", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1790000000000001e-05 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898773+0000", 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000106945 2024-09-17T21:49:07.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.898783+0000", 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1290000000000002e-06 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.901715+0000", 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0029321299999999998 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.901727+0000", 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.185e-05 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:28875 3.4 3.d0df08c4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:49:06.902020+0000", 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.17411823400000001, 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.174186323, 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:49:07.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 28875 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902020+0000", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902020+0000", 2024-09-17T21:49:07.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902022+0000", 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.12e-06 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902030+0000", 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.7649999999999993e-06 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902043+0000", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3010000000000001e-05 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902052+0000", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4849999999999998e-06 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902107+0000", 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4123999999999999e-05 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:06.902121+0000", 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4103e-05 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 2 2024-09-17T21:49:07.087 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:07.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:07.130 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.126+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.231 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:07.229+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:07.236 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:07.236 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:07.236 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:07.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:07.332 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.329+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.268642284999999, 2024-09-17T21:49:07.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:07.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:07.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267483952000006, 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.361 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267467406999998, 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:07.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:07.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:07.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267452641999995, 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:07.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:07.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267425337000006, 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.376 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:07.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267388448999995, 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.381 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:07.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267349750999998, 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:07.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:07.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:07.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267311578000005, 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:07.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.267023237999993, 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:07.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:07.400 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.266996641999995, 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:07.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:07.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.692458836, 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:07.405 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:07.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:07.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.692425520999997, 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:07.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.420 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.685335457000001, 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.421 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:07.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:07.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:07.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:07.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.685301776999999, 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:07.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.596832847000002, 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:07.434 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:07.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:07.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.596739827999997, 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:07.442 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:07.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:07.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:07.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.594109881000001, 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:07.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:07.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:07.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:07.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.459 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 37.250567177999997, 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.460 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.462 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:07.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.595247681000004, 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:07.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:07.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:07.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:07.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:07.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:07.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:07.474 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 36.589338650999998, 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.475 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.476 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:07.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:07.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.481 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:07.483 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:07.484 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.480+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.584 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.581+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.634 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.634 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:07.634 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:07.634 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:07.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:07.685 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:07.683+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:07.786 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.783+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.789 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.789 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:07.790 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:07.790 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:07.790 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:07.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:07.887 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:07.883+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:07.951 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:07.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 153.07550097999999, 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:07.952 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:07.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:07.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.039054669999999, 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.958 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:07.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.039033368999995, 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.963 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:07.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.039012607999993, 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:07.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:07.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:07.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038991502000002, 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:07.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:07.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:07.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038971012999994, 2024-09-17T21:49:07.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:07.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:07.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038946916, 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:07.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038924422999997, 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038896816000005, 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:07.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.038871975999996, 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:07.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:07.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:08.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.886918201, 2024-09-17T21:49:08.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:08.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.886894439000002, 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:08.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:08.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.886852050000002, 2024-09-17T21:49:08.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:08.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:08.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:08.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:08.016 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.831103397000007, 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:08.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.831080302000004, 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.021 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:08.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.025 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.831053921999995, 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:08.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.831037285999997, 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:08.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.831007322000005, 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.035 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:08.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.830994396999998, 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:08.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 76.830984176000001, 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:08.044 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:08.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:08.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:08.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:08.050 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:08.047+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:08.150 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:08.147+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:08.170 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:08.171 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:08.171 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:08.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:08.263 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:08.261+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:08.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:08.328 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:08.328 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:08.328 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:08.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:08.352 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:08.349+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:08.434 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:08.451 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:08.451 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:08.453 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:08.451+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:08.467 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:08.467 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:08.485 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:08.485 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:08.487 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:08.487 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:08.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:08.487 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 162.744883942, 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:08.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:08.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:08.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 162.743839482, 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:08.493 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:08.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:08.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:08.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.498 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.54265415099999, 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:08.499 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:08.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:08.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:08.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.326736483000005, 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.506 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:08.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.124299741000002, 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.511 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:08.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:08.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:08.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.517 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.124242561000003, 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:08.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:08.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:08.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:08.524 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123671979999997, 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:08.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:08.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123644045999995, 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:08.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:08.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123509694000006, 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:08.535 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:08.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123488159000004, 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:08.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:08.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:08.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123327193999998, 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:08.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:08.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.123271428999999, 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:08.552 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:08.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.121924821999997, 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:08.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.121873265999994, 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:08.563 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:08.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:08.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:08.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.121185005000001, 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.569 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:08.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:08.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.121167954000001, 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:08.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.120229714000004, 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:08.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:08.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.120199373999995, 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:08.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:08.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.825801925999997, 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:08.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:08.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.924684225999997, 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:08.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:08.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:08.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:08.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:08.602 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:08.603 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:08.603 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:08.604 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:08.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:08.605 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:08.601+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:08.605 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:08.704 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:08.701+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:08.753 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:08.753 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:08.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:08.805 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:08.802+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:08.868 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:08.868 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:08.877 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:08.905 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:08.902+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:08.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:49:08.989 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:08.989 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:08.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:09.006 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:09.003+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:09.107 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:09.104+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:09.113 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.113 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:09.208 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:09.206+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:09.236 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.236 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.246 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:09.329 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:09.327+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:09.392 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.393 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:09.445 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:09.442+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:09.510 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:09.507+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:09.522 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.522 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:09.611 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:09.609+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:09.683 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.683 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.693 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:09.736 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:09.733+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:09.813 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.813 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:09.813 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:09.810+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:09.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:09.908 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:49:09.917 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:09.913+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:09.993 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:09.994 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:10.048 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:10.045+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:10.115 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:10.114+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:10.120 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.120 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:10.217 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:10.215+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:10.307 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.307 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:10.318 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:10.315+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:10.419 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:10.417+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:10.437 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.437 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.448 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:10.519 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:10.518+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.585 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:10.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:10.620 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:10.618+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:10.721 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:10.718+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:10.722 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.722 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.732 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:10.793 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:10.810 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:10.811 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:10.822 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:10.819+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:10.836 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:10.836 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:10.853 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:10.854 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:10.855 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:10.855 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:10.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:10.874 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:10.874 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:10.895 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:10.896 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:10.947 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:10.945+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:11.020 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:11.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:11.020 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:11.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:11.020 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:11.023 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:11.021+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:11.029 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:11.124 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:11.121+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 165.435468284, 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:11.178 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:11.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 165.43442382399999, 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:11.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:11.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:11.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 104.233238493, 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:11.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:11.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:11.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:11.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.017320824999999, 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:11.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:11.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:11.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814884082999995, 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:11.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:11.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814826902999997, 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.212 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:11.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:11.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814256322000006, 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814228388000004, 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:11.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:11.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814094036, 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:11.230 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:11.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:11.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.236 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.814072500999998, 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:11.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:11.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:11.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:11.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.813911536000006, 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:11.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:11.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:11.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.813855770999993, 2024-09-17T21:49:11.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:11.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.812509164000005, 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:11.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.812457608000003, 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:11.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:11.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.811769346999995, 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:11.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.811752295999995, 2024-09-17T21:49:11.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:11.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.810814055999998, 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:11.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.810783716000003, 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.284 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:11.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:11.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:11.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.516386268000005, 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:11.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:11.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:11.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:11.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 66.615268568000005, 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:11.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:11.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:11.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.306 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:11.308 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:11.308 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:11.305+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:11.408 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:11.406+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:11.456 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:11.457 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:11.457 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:11.457 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:11.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:11.509 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:11.506+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:11.609 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:11.607+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:11.612 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:11.612 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:11.613 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:11.613 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:11.613 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:11.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:11.711 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:11.708+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:11.770 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79664365799999, 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:11.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79661939799999, 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:11.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:11.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:11.780 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79659351500001, 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:11.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.796568514, 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:11.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.789 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79652871499999, 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:11.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79651297000001, 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:11.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79650143500001, 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:11.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.796491135, 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:11.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:11.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79451989200001, 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:11.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:11.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79450674399999, 2024-09-17T21:49:11.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:11.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:11.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.794496138, 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:11.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:11.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79448579999999, 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:11.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:11.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.794475793, 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:11.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79446584199999, 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.835 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79445053399999, 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:11.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:11.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.79246369099999, 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.792451202, 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:11.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:11.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 174.792440739, 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:11.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.093104941, 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.854 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:11.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:11.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.738374965, 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:11.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:11.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:11.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:11.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:11.864 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:11.865 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:11.866 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:11.862+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:11.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:49:11.932 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:11.932 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:11.932 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-17T21:49:11.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-17T21:49:11.966 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:11.963+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:12.019 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:12.020 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:12.028 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:12.069 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:12.067+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:12.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-17T21:49:12.167 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:12.165+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:12.175 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:12.175 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:12.176 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:12.176 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:12.176 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:12.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:12.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-17T21:49:12.268 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:12.265+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:12.333 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358486194999998, 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.334 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:12.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:12.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.340 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358462058000001, 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:12.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:12.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358429803, 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.349 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:12.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:12.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.355 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358399749999997, 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.356 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:12.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358376403999998, 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358323456999997, 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:12.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.370 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.358290111000002, 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:12.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354958877999998, 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.376 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:12.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354930977999999, 2024-09-17T21:49:12.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.386 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354908272000003, 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.387 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:12.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354892004, 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:12.392 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:12.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:12.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354882033000003, 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:12.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354869884000003, 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:12.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:12.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354860189, 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:12.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:12.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354850595999999, 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.413 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354841106000002, 2024-09-17T21:49:12.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:12.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:12.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.424 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354831259999997, 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:12.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:12.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.354821919999999, 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:12.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:12.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.656608506000001, 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:12.435 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:12.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:12.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:12.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.656018534000001, 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:12.442 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:12.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:12.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:12.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:12.448 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:12.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-17T21:49:12.450 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:12.447+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:12.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:12.550 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:12.547+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:12.602 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:12.602 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:12.602 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:12.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:12.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-17T21:49:12.650 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:12.649+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:12.719 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:12.720 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:12.720 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:12.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:12.751 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:12.750+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:12.852 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:12.850+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.855 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.16888699, 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:12.857 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.858 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:12.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:12.861 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.136612015, 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.862 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.863 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:12.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:12.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.13637324699999, 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.868 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.869 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.870 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:12.871 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.872 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.968867117, 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.873 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.874 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.875 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:12.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:12.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.961965346, 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.879 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:12.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:12.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.961849255, 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:12.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:12.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:12.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:12.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.764845734999994, 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.890 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:12.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.76480694, 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:12.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:12.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:12.900 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.764792655999997, 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.901 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:12.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.764547690000001, 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.906 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:12.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:12.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:12.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.764465242, 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:12.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:12.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.764433432000004, 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:12.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:12.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.377770315999996, 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:12.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:12.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:12.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:12.934 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.367841335999998, 2024-09-17T21:49:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:12.935 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.936 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.175506622, 2024-09-17T21:49:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:12.949 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:12.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:12.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:12.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:12.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:12.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.175249201, 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:12.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.175216355000003, 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:12.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:12.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.175199410000005, 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:12.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:12.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:12.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 65.175157913000007, 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:12.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:12.988 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 19.18092141, 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:12.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:12.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:12.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:12.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:12.994 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:12.994 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:12.996 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:12.993+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:12.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:12.997 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:49:13.096 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:13.093+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:13.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:13.150 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:13.150 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:13.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:13.165 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:13.165 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:13.182 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:13.182 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:13.197 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:13.194+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:13.199 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:13.199 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:13.216 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:13.216 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:13.233 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:13.233 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:13.250 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:13.274 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.274 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:13.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:13.299 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:13.297+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:13.400 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:13.396+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:13.431 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.431 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:13.440 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:13.500 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:13.498+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:13.588 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.588 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:13.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:13.640 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:13.637+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:13.703 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:13.699+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:13.713 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.713 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:13.723 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:13.803 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:13.801+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:13.873 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.874 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:13.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:13.926 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:13.924+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:13.993 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:13.993 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:14.043 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:14.040+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:14.105 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:14.103+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:14.149 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.152 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:14.206 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:14.204+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:14.306 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.307 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:14.304+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:14.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:14.408 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:14.405+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:14.463 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.463 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:14.508 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:14.506+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:14.609 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:14.606+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:14.618 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.618 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.627 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:14.710 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:14.707+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:14.744 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.744 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.754 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:14.811 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:14.808+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:14.865 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:14.866 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:14.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:14.915 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:14.913+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:14.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:49:15.012 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:15.010+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:15.021 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:15.021 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:15.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:15.113 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:15.110+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:15.178 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:15.178 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:15.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:15.230 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:15.227+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:15.264 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:15.295 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:15.304 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:15.315 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:15.313+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:15.389 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:15.406 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:15.416 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:15.414+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:15.424 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:15.424 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:15.440 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:15.441 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:15.454 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:15.454 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:15.455 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:15.455 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:15.457 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:15.457 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:15.465 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:15.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:15.477 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:15.517 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:15.515+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:15.585 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 173.89902754600001, 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.586 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:15.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 173.86675257100001, 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.592 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.597 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 173.866513803, 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:15.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.699007673, 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.603 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:15.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:15.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.69210590199999, 2024-09-17T21:49:15.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:15.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:15.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.691989811, 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:15.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:15.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494986291000004, 2024-09-17T21:49:15.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:15.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.625 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494947495999995, 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:15.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:15.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494933212000007, 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:15.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:15.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:15.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494688245999996, 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.636 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:15.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494605797999995, 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:15.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.494573987999999, 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.647 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:15.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.651 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.107910872000005, 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:15.652 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:15.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.097981892000007, 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.905647177999995, 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.664 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:15.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:15.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.905389756999995, 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:15.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:15.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.905356910999998, 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:15.681 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:15.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.905339966, 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.687 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:15.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:15.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:15.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:15.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.905298469000002, 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:15.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:15.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 21.911061965999998, 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:15.700 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:15.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:15.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:15.705 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:15.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:49:15.708 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:15.703+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:15.808 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:15.805+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:15.862 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:15.862 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:15.862 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:15.862 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:15.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:15.914 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:15.911+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:15.978 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:15.978 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:15.978 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:15.987 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:16.009 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:16.006+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:16.110 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:16.106+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.049755246999993, 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:16.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:16.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048596914000001, 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.143 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:16.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:16.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048580369000007, 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:16.148 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:16.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048565604000004, 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:16.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:16.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048538299000001, 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:16.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:16.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048501411000004, 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048462713000006, 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:16.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048424539999999, 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.175 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048136200000002, 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:16.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:16.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.185 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.048109604000004, 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:16.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:16.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.473571797999995, 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:16.191 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:16.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:16.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.473538482999999, 2024-09-17T21:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.466448419000002, 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 72.466414739000001, 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.377945809000003, 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:16.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:16.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:16.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:16.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:16.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:16.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.377852789999999, 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:16.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:16.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:16.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.375222843000003, 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:16.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:16.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:16.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:16.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 46.031680139999999, 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:16.245 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:16.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:16.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.376360642999998, 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:16.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:16.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:16.258 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 45.370451613, 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:16.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.264 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:16.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:16.267 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:16.264+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:16.367 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:16.365+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:16.414 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:16.414 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:16.414 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:16.423 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:16.468 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:16.464+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:16.530 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:16.530 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:16.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:16.569 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:16.567+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:16.670 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:16.667+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:16.695 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 161.818483606, 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:16.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:16.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:16.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.701 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.782037295999999, 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:16.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.782015994999995, 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:16.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:16.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781995233999993, 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:16.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:16.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781974128000002, 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.716 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:16.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:16.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781953638999994, 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:16.720 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:16.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:16.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:16.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781929542, 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:16.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781907048999997, 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781879442000005, 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.738 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 89.781854601999996, 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:16.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.629900827, 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:16.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.629877065000002, 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:16.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.629834676000002, 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.752 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:16.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.756 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.574086023000007, 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:16.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.574062928000004, 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.761 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:16.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:16.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.574036547999995, 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.765 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:16.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:16.769 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.574019911999997, 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:16.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.573989948000005, 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.774 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:16.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.573977022999998, 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:16.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 85.573966802000001, 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:16.783 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:16.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:16.787 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:16.788 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:16.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:16.790 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:16.786+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:16.889 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:16.886+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:16.905 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:16.905 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:16.905 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:16.905 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:16.914 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:16.990 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:16.987+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:17.025 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:17.025 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:17.025 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:17.025 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:17.025 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:17.030 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:17.091 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:17.088+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:17.180 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 171.43736434100001, 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:17.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 171.436319881, 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:17.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 110.23513455, 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:17.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.019216881999995, 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:17.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:17.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.816780140000006, 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:17.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:17.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:17.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.816722960000007, 2024-09-17T21:49:17.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:17.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:17.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:17.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.816152379000002, 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:17.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:17.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:17.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.816124445, 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:17.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.815990092999996, 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:17.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:17.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.815968557999994, 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:17.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:17.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.815807593000002, 2024-09-17T21:49:17.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:17.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.815751828000003, 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:17.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.814405221000001, 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.252 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:17.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.257 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.814353664999999, 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:17.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:17.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.813665404000005, 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:17.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.813648353000005, 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:17.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.812710112999994, 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:17.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:17.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:17.279 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.812679772999999, 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:17.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.518282325000001, 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:17.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:17.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:17.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:17.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.617164625000001, 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:17.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.297 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:17.299 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:17.299 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:17.295+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:17.400 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:17.396+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:17.407 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:17.407 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:17.407 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:17.407 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:17.416 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:17.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:17.522 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:17.522 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:17.522 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:17.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:17.533 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:17.530+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:17.601 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:17.598+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:17.618 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:17.619 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:17.635 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:17.636 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:17.639 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66557285600001, 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66554859600001, 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.665522713, 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66549771199999, 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66545791300001, 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.665442168, 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.665430633, 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66542033299999, 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66344909, 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66343594200001, 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66342533599999, 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66341499800001, 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66340499099999, 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66339504000001, 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66337973200001, 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:17.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66139288900001, 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:17.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:17.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66138040000001, 2024-09-17T21:49:17.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:17.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:17.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 180.66136993699999, 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:17.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:17.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:17.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.962034139, 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:17.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.607304162999998, 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:17.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:17.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:17.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:17.731 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:17.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:49:17.732 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:17.733 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:17.733 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:17.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:17.734 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:17.734 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:17.729+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:17.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:17.735 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:17.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:17.736 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:17.828 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2] out_osds: [3] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:17.828 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:17.829 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-17T21:49:17.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-17T21:49:17.833 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:17.830+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:17.884 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:17.884 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:17.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:17.934 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:17.931+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:18.034 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:18.031+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:18.039 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.040 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.048 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:18.133 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-17T21:49:18.130+0000 7f0c28614640 -1 mon.a@0(leader).osd e195 definitely_dead 0 2024-09-17T21:49:18.136 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:18.133+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:18.155 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.155 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.164 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:18.237 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:18.234+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:18.272 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.273 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:18.338 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:18.335+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:18.439 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:18.436+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:18.504 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.505 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:18.539 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:18.535+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:18.620 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.621 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:18.673 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:18.670+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:18.688 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:49:18.691 INFO:teuthology.orchestra.run.smithi007.stderr:marked in osd.3. 2024-09-17T21:49:18.713 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-17T21:49:18.745 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.745 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.746 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:18.742+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:18.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:18.842 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:18.838+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:18.903 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:18.903 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:18.913 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:18.943 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:18.940+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:19.044 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:19.041+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:19.059 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.060 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:19.153 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:19.150+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.215 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.224 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:19.270 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:19.262+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:19.346 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:19.343+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.376 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:19.447 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:19.443+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:19.506 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.506 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:19.557 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:19.554+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:19.648 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:19.645+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:19.665 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.665 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.677 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:19.737 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:19.781 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:19.778+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:19.787 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.787 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:19.849 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:19.846+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:19.860 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:19.860 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:19.877 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:19.877 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:19.893 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:19.904 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:19.904 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:19.909 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:19.910 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:19.913 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:19.926 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:19.927 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:19.943 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:19.944 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:19.951 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:19.947+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:20.052 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:20.048+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:20.061 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:20.061 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:20.061 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:20.061 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:20.061 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:20.070 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:20.153 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:20.150+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:20.218 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24457891, 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24455465, 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24452876699999, 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:20.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:20.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:20.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:20.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24450376600001, 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:20.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.244463967, 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24444822199999, 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24443668699999, 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.246 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:20.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.244426387, 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:20.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:20.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24245514399999, 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.242441996, 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:20.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24243139000001, 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:20.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.242421052, 2024-09-17T21:49:20.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:20.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:20.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24241104500001, 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.242401094, 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:20.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.242385786, 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:20.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:20.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:20.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.240398943, 2024-09-17T21:49:20.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:20.287 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:20.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.240386454, 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 183.24037599100001, 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:20.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.541040193000001, 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:20.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.186310216999999, 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:20.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.309 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:20.311 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:20.311 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:20.307+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:20.412 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:20.408+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:20.459 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:20.459 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:20.459 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:20.459 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:20.467 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:20.551 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:20.549+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:20.614 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:20.611+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:20.614 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:20.614 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:20.615 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:20.615 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:20.623 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:20.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:49:20.715 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:20.713+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:20.794 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819569639000001, 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:20.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819545501999997, 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.800 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:20.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.804 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819513247000003, 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:20.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:20.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:20.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819483194, 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:20.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:20.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819459848000001, 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:20.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:20.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:20.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819406901000001, 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.819 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:20.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:20.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.819373554999999, 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:20.823 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:20.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.816042322000001, 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:20.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.816014422000002, 2024-09-17T21:49:20.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:20.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815991715999999, 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:20.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:20.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815975448000003, 2024-09-17T21:49:20.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:20.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815965476999999, 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:20.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815953327999999, 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:20.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:20.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815943633000003, 2024-09-17T21:49:20.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:20.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815934040000002, 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:20.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:20.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815924549999998, 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.865 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:20.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:20.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815914704000001, 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:20.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:20.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.815905364000002, 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:20.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.117691950000001, 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:20.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.117101978000001, 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:20.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:20.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:20.888 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:20.890 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:20.887+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:20.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:20.990 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:20.987+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:21.003 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:21.003 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:21.003 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:21.092 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:21.089+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:21.162 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.162 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:21.162 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:21.162 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:21.162 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:21.214 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:21.211+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:21.280 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:21.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 179.59396580200001, 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:21.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:21.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:21.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 179.56169082700001, 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 179.561452059, 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:21.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:21.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:21.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 130.39394592900001, 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 130.38704415800001, 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 130.38692806700001, 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:21.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:21.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189924547000004, 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.313 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:21.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:21.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189885751999995, 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:21.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.322 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189871468000007, 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:21.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189626501999996, 2024-09-17T21:49:21.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189544053999995, 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:21.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.189512243999999, 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.337 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:21.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:21.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:21.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 91.802849128000005, 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:21.342 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:21.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:21.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 91.792920147999993, 2024-09-17T21:49:21.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:21.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.600585433999996, 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:21.353 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:21.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:21.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.600328012999995, 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:21.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:21.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:21.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.365 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.600295166999999, 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:21.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:21.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:21.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.371 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.600278222, 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:21.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:21.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.600236725000002, 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:21.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:21.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 27.606000221999999, 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:21.384 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:21.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.389 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.391 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:21.391 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:21.387+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:21.491 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:21.488+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.541 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:21.541 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:21.541 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:21.592 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:21.589+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:21.693 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:21.689+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:21.700 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.700 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:21.700 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:21.700 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:21.700 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.707 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:49:21.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:21.811 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:21.808+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:21.883 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.796591848000006, 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.884 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:21.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:21.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:21.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795433514999999, 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:21.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:21.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795416970000005, 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:21.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.898 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795402205000002, 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:21.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:21.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795374899999999, 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:21.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:21.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795338012000002, 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:21.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:21.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795299314000005, 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:21.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.795261140999997, 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.918 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:21.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.794972801, 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:21.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:21.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 83.794946205000002, 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:21.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:21.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.220408398999993, 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:21.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:21.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:21.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.220375083999997, 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:21.938 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.213285020000001, 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.944 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:21.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 78.213251339999999, 2024-09-17T21:49:21.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:21.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:21.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.124782410000002, 2024-09-17T21:49:21.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:21.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:21.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.124689390999997, 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:21.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:21.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 52.122059444000001, 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:21.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:21.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:21.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.778516740999997, 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.975 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.123197244000004, 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.981 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:21.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:21.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:21.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 51.117288213999998, 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:21.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:21.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:21.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:21.992 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:21.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:21.994 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:21.991+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:21.995 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:22.094 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:22.092+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:22.150 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:22.150 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:22.150 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:22.150 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:22.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:22.169 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:22.169 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:22.188 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:22.195 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:22.193+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:22.207 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:22.207 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:22.227 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:22.227 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:22.246 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:22.266 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:22.267 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:22.275 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.275 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:22.326 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:22.323+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:22.398 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:22.395+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:22.445 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.445 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:22.498 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:22.496+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:22.569 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.569 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.579 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:22.599 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:22.596+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:22.700 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:22.696+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:22.743 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.743 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:22.801 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:22.799+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:22.860 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.860 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:22.902 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:22.899+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:22.976 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:22.976 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:22.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:23.028 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:23.026+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:23.092 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.092 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:23.142 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:23.139+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:23.204 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:23.201+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:23.247 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.247 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.256 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:23.305 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:23.302+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:23.402 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.403 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.406 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:23.403+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:23.412 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:23.506 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:23.504+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:23.518 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.518 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.527 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:23.607 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:23.605+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:23.634 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.634 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:23.708 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:23.705+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:23.715 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:23.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:49:23.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:49:23.777 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.777 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:23.810 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:23.807+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:23.910 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:23.907+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:23.939 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:23.939 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:23.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:24.033 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:24.030+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:24.096 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:24.096 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:24.105 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:24.146 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:24.144+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:24.213 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:24.210+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:24.253 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:24.253 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:24.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:24.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:24.345 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:24.342+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:24.412 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:24.412 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:24.414 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:24.411+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:24.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:24.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:24.426 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:24.444 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:24.444 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:24.460 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:24.461 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:24.478 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:24.478 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:24.495 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:24.495 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:24.513 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:24.513 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:24.515 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:24.512+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:24.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.450691547000005, 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:24.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449533213999999, 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:24.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449516669000005, 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:24.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:24.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449501904000002, 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:24.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:24.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:24.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449474598999998, 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:24.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449437711000002, 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:24.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:24.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449399013000004, 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.567 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:24.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:24.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.571 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449360839999997, 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:24.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.4490725, 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.576 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:24.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:24.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.580 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.449045904000002, 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:24.581 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:24.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:24.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:24.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.874508098000007, 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:24.590 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:24.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:24.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.597 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.874474782999997, 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:24.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:24.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.867384719, 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:24.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:24.609 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 80.867351038999999, 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:24.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:24.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:24.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.778882109000001, 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.615 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:24.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:24.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:24.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.778789089999997, 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:24.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:24.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.629 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.776159143000001, 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:24.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:24.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:24.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:24.638 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 54.432616439999997, 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:24.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:24.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.777296943000003, 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:24.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:24.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:24.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 53.771387912999998, 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:24.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:24.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:24.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:24.657 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:24.658 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:24.659 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:24.656+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:49:24.716 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:49:24.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:49:24.759 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:24.756+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:24.816 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:24.817 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:24.817 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:24.817 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:24.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:24.859 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:24.857+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:24.961 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:24.958+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:24.975 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:24.975 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:24.975 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:24.975 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:24.975 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:24.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:25.068 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:25.066+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 170.25828724900001, 2024-09-17T21:49:25.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:25.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:25.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:25.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:25.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221840939000003, 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:25.144 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221819637999999, 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.145 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:25.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.149 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221798876999998, 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221777771000006, 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:25.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221757281999999, 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.159 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:25.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:25.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221733185000005, 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.163 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:25.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:25.167 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221710692000002, 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:25.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:25.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221683084999995, 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:25.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:25.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:25.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 98.221658245, 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.176 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:25.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:25.179 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.069704470000005, 2024-09-17T21:49:25.180 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:25.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:25.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.069680708000007, 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.069638319000006, 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:25.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013889665999997, 2024-09-17T21:49:25.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:25.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:25.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013866570999994, 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.198 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:25.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.202 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013840191, 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:25.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:25.207 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013823555000002, 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:25.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013793590999995, 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.212 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:25.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013780666000002, 2024-09-17T21:49:25.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:25.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.013770445000006, 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:25.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:25.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:25.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:25.225 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:25.227 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:25.227 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:25.224+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:25.327 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:25.324+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:25.338 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:25.338 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:25.339 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:25.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:25.431 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:25.429+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:25.494 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:25.494 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:25.494 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:25.495 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:25.495 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:25.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:25.529 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:25.526+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:25.629 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:25.627+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:25.652 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:25.652 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 179.91002705299999, 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:25.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 179.90898259299999, 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:25.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:25.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:25.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:25.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 118.707797262, 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:25.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:25.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:25.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.491879593999997, 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:25.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:25.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.289442851999993, 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:25.676 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:25.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:25.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.289385671999995, 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:25.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:25.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:25.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:25.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288815091000004, 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:25.690 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:25.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:25.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:25.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288787157000002, 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288652804999998, 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:25.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:25.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:25.712 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288631269999996, 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:25.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:25.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288470305000004, 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:25.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:25.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:25.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:25.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:25.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.288414540000005, 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:25.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.287067933000003, 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:25.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:25.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.287016377, 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:25.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:25.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.286328116000007, 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:25.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:25.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:25.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.286311065000007, 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:25.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.285372824999996, 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:25.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.285342485000001, 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:25.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.990945037000003, 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:25.772 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:25.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:25.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:25.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.089827337000003, 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:25.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:25.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:25.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:25.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:25.785 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:25.787 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:49:25.788 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:25.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:49:25.789 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:25.784+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:25.888 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:25.885+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:25.902 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:25.902 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:25.902 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:25.902 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:25.911 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:25.989 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:25.987+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:26.057 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:26.057 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:26.057 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:26.066 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:26.089 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:26.087+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.200887411, 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:26.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.20086315099999, 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:26.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:26.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:26.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.20083726799999, 2024-09-17T21:49:26.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:26.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:26.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.200812267, 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:26.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:26.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.200772468, 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.193 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.20075672300001, 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:26.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.20074518800001, 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:26.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:26.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.200734888, 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:26.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.19876364500001, 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:26.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.19875049699999, 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:26.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:26.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.198739891, 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.19872955299999, 2024-09-17T21:49:26.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:26.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:26.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.19871954600001, 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:26.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:26.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.198709595, 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.198694287, 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.196707444, 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:26.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:26.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.196694955, 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:26.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:26.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 189.196684492, 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.497348694000003, 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:26.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:26.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:26.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.142618718000001, 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:26.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:26.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:26.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.264 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.265 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:26.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:26.267 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:26.264+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:26.367 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:26.365+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:26.373 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:26.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:26.468 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:26.465+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:26.489 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:26.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:26.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:26.570 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:26.568+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.646 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671238946999999, 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:26.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671214810000002, 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671182555000001, 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:26.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671152501999998, 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:26.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:26.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671129155999999, 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671076208999999, 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:26.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:26.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.671042862999997, 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667711629999999, 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.66768373, 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:26.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667661023999997, 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:26.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667644756000001, 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:26.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:26.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667634784999997, 2024-09-17T21:49:26.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:26.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:26.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667622635999997, 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:26.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667612941000002, 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:26.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:26.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667603348, 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:26.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:26.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667593858000004, 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:26.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:26.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667584011999999, 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:26.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:26.723 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.667574672000001, 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:26.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.969361257999999, 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:26.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.968771285999999, 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:26.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:26.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:26.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:26.739 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:26.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:49:26.740 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:26.741 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:26.741 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:26.742 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:26.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:26.743 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:26.743 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:26.743 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:26.744 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:26.744 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:49:26.745 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:26.738+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:26.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:49:26.746 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:26.746 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:26.841 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:26.839+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:26.871 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:26.871 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:26.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:26.965 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:26.963+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:27.030 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.030 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.039 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:27.080 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:27.078+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:27.143 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:27.140+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:27.187 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.187 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.196 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:27.244 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:27.242+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:27.302 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.303 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.311 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:27.345 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:27.343+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:27.419 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.419 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:27.471 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:27.469+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:27.535 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.535 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:27.585 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:27.583+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:27.648 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:27.646+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.692 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:27.730 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:49:27.731 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:49:27.748 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:27.746+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:27.752 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:49:27.849 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.849 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.849 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:27.846+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:27.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:27.950 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:27.948+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:27.970 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:27.970 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:27.979 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:28.051 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:28.049+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:28.089 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.089 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:28.152 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:28.149+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:28.204 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.205 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:28.257 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:28.255+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:28.320 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.321 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:28.353 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:28.351+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:28.454 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:28.452+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.477 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:28.569 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:28.567+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.633 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.642 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:28.656 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:28.654+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:28.674 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:49:28.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:49:28.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:28.756 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:28.754+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:28.801 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:28.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:28.857 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:28.855+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:28.908 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:28.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:28.925 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:28.926 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:28.942 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:28.942 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:28.957 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:28.955+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:28.959 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:28.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:28.960 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:28.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:28.975 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:28.976 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:28.992 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:28.993 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:29.058 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:29.056+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:29.075 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:29.083 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:29.160 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:29.157+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:29.191 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216599043000002, 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:29.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216574905999998, 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:29.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:29.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.201 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216542650999997, 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:29.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:29.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216512598000001, 2024-09-17T21:49:29.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:29.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:29.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216489252000002, 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:29.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:29.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216436305000002, 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:29.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:29.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.216402959, 2024-09-17T21:49:29.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:29.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:29.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.213071726000003, 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:29.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.229 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.213043826000003, 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.234 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.213021120000001, 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:29.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.213004851999997, 2024-09-17T21:49:29.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.243 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212994881, 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:29.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:29.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212982732, 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:29.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:29.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212973036999998, 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:29.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212963444000003, 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:29.258 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:29.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212953954, 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:29.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212944108000002, 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:29.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.212934767999997, 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:29.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:29.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.514721354000002, 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:29.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:29.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.514131382000002, 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:29.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:29.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:29.288 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:29.289 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:29.286+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:29.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:29.390 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:29.388+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:29.399 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:29.399 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:29.399 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:29.408 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:29.492 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:29.489+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:29.514 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:29.515 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:29.515 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:29.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:29.593 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:29.590+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:29.632 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:29.632 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 187.945963947, 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:29.633 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.635 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:29.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 187.91368897199999, 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:29.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 187.91345020399999, 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:29.645 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:29.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:29.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 138.74594407399999, 2024-09-17T21:49:29.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:29.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:29.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:29.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:29.656 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 138.73904230299999, 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:29.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 138.738926212, 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:29.662 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.541922692, 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.667 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.668 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:29.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:29.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.54188389700001, 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:29.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:29.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.541869613, 2024-09-17T21:49:29.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.54162464700001, 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:29.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:29.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.54154219900001, 2024-09-17T21:49:29.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:29.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 104.541510389, 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:29.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.154847273, 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.697 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:29.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.699 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:29.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.144918293, 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:29.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:29.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:29.710 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.952583579000006, 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:29.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:29.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:29.717 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.952326158000005, 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:29.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:29.723 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.952293311999995, 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:29.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.729 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.952276366999996, 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:29.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:29.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.735 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.952234869999998, 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.736 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:29.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:29.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 35.957998367000002, 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:29.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:29.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:29.747 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:29.749 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:29.749 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:49:29.750 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:29.747+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:29.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:49:29.757 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:49:29.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:49:29.885 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:29.883+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:29.917 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:29.917 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:29.917 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:29.927 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:29.950 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:29.948+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:30.052 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:30.049+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:30.074 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:30.074 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:30.075 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:30.152 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:30.150+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:30.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:30.180 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:49:30.192 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-17T21:49:30.137610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1566076587,"num_objects":1844,"num_object_clones":295,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":894,"num_read":371595,"num_read_kb":1867932,"num_write":21080,"num_write_kb":604272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10866,"num_bytes_recovered":1234485419,"num_keys_recovered":2095,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18743,"ondisk_log_size":18743,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1004200,"kb_used_data":681488,"kb_used_omap":36,"kb_used_meta":322203,"kb_avail":565226840,"statfs":{"total":579820584960,"available":578792284160,"internally_reserved":0,"allocated":697843712,"data_stored":997969306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":329936169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-77031375,"num_objects":-109,"num_object_clones":-37,"num_object_copies":-218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-109,"num_whiteouts":-32,"num_read":567,"num_read_kb":478,"num_write":117,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":69395503,"num_keys_recovered":95,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000683"},"pg_stats":[{"pgid":"3.1f","version":"202'353","reported_seq":11463,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722437+0000","last_change":"2024-09-17T21:49:25.723051+0000","last_active":"2024-09-17T21:49:25.722437+0000","last_peered":"2024-09-17T21:49:25.722437+0000","last_clean":"2024-09-17T21:49:25.722437+0000","last_became_active":"2024-09-17T21:49:23.715014+0000","last_became_peered":"2024-09-17T21:49:23.715014+0000","last_unstale":"2024-09-17T21:49:25.722437+0000","last_undegraded":"2024-09-17T21:49:25.722437+0000","last_fullsized":"2024-09-17T21:49:25.722437+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:45.314185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040224290000000001,"stat_sum":{"num_bytes":51420875,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":11121,"num_read_kb":70761,"num_write":333,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1e","version":"203'512","reported_seq":20104,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726370+0000","last_change":"2024-09-17T21:49:25.726370+0000","last_active":"2024-09-17T21:49:25.726370+0000","last_peered":"2024-09-17T21:49:25.726370+0000","last_clean":"2024-09-17T21:49:25.726370+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:49:25.726370+0000","last_undegraded":"2024-09-17T21:49:25.726370+0000","last_fullsized":"2024-09-17T21:49:25.726370+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:03.506844+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0069220180000000003,"stat_sum":{"num_bytes":106936864,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":18711,"num_read_kb":109023,"num_write":400,"num_write_kb":34487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1d","version":"203'377","reported_seq":5221,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.725277+0000","last_change":"2024-09-17T21:49:25.725740+0000","last_active":"2024-09-17T21:49:25.725277+0000","last_peered":"2024-09-17T21:49:25.725277+0000","last_clean":"2024-09-17T21:49:25.725277+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:49:25.725277+0000","last_undegraded":"2024-09-17T21:49:25.725277+0000","last_fullsized":"2024-09-17T21:49:25.725277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:33.608857+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0035601869999999998,"stat_sum":{"num_bytes":34531401,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":4895,"num_read_kb":51120,"num_write":639,"num_write_kb":31818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1c","version":"203'558","reported_seq":18089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.759849+0000","last_change":"2024-09-17T21:49:25.727050+0000","last_active":"2024-09-17T21:49:25.759849+0000","last_peered":"2024-09-17T21:49:25.759849+0000","last_clean":"2024-09-17T21:49:25.759849+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:49:25.759849+0000","last_undegraded":"2024-09-17T21:49:25.759849+0000","last_fullsized":"2024-09-17T21:49:25.759849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:38:28.084682+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.004765895,"stat_sum":{"num_bytes":74653797,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":26,"num_read":19621,"num_read_kb":86798,"num_write":606,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1b","version":"202'426","reported_seq":16696,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.720426+0000","last_change":"2024-09-17T21:49:25.720426+0000","last_active":"2024-09-17T21:49:25.720426+0000","last_peered":"2024-09-17T21:49:25.720426+0000","last_clean":"2024-09-17T21:49:25.720426+0000","last_became_active":"2024-09-17T21:48:08.675352+0000","last_became_peered":"2024-09-17T21:48:08.675352+0000","last_unstale":"2024-09-17T21:49:25.720426+0000","last_undegraded":"2024-09-17T21:49:25.720426+0000","last_fullsized":"2024-09-17T21:49:25.720426+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:18:02.740360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00165318,"stat_sum":{"num_bytes":66587664,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":14405,"num_read_kb":43490,"num_write":345,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1a","version":"202'623","reported_seq":11331,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.842681+0000","last_change":"2024-09-17T21:49:24.717627+0000","last_active":"2024-09-17T21:49:24.842681+0000","last_peered":"2024-09-17T21:49:24.842681+0000","last_clean":"2024-09-17T21:49:24.842681+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:49:24.842681+0000","last_undegraded":"2024-09-17T21:49:24.842681+0000","last_fullsized":"2024-09-17T21:49:24.842681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:46.986734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010175710000000001,"stat_sum":{"num_bytes":98333436,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":13366,"num_read_kb":81957,"num_write":660,"num_write_kb":25541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"3.19","version":"206'397","reported_seq":4818,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.764000+0000","last_change":"2024-09-17T21:49:28.672321+0000","last_active":"2024-09-17T21:49:28.764000+0000","last_peered":"2024-09-17T21:49:28.764000+0000","last_clean":"2024-09-17T21:49:28.764000+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:49:28.764000+0000","last_undegraded":"2024-09-17T21:49:28.764000+0000","last_fullsized":"2024-09-17T21:49:28.764000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:54.013206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666739,"stat_sum":{"num_bytes":33117314,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":4426,"num_read_kb":33920,"num_write":402,"num_write_kb":13444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.18","version":"203'616","reported_seq":12855,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.721818+0000","last_change":"2024-09-17T21:49:25.722167+0000","last_active":"2024-09-17T21:49:25.721818+0000","last_peered":"2024-09-17T21:49:25.721818+0000","last_clean":"2024-09-17T21:49:25.721818+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:49:25.721818+0000","last_undegraded":"2024-09-17T21:49:25.721818+0000","last_fullsized":"2024-09-17T21:49:25.721818+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:33:02.344566+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003434327,"stat_sum":{"num_bytes":77495033,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":12050,"num_read_kb":68756,"num_write":731,"num_write_kb":13790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.17","version":"196'358","reported_seq":7544,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.719438+0000","last_change":"2024-09-17T21:49:25.720393+0000","last_active":"2024-09-17T21:49:25.719438+0000","last_peered":"2024-09-17T21:49:25.719438+0000","last_clean":"2024-09-17T21:49:25.719438+0000","last_became_active":"2024-09-17T21:48:10.691486+0000","last_became_peered":"2024-09-17T21:48:10.691486+0000","last_unstale":"2024-09-17T21:49:25.719438+0000","last_undegraded":"2024-09-17T21:49:25.719438+0000","last_fullsized":"2024-09-17T21:49:25.719438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:41.918512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012261640000000001,"stat_sum":{"num_bytes":19398144,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":7903,"num_read_kb":32043,"num_write":336,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.16","version":"202'934","reported_seq":11533,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726819+0000","last_change":"2024-09-17T21:49:25.727071+0000","last_active":"2024-09-17T21:49:25.726819+0000","last_peered":"2024-09-17T21:49:25.726819+0000","last_clean":"2024-09-17T21:49:25.726819+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:49:25.726819+0000","last_undegraded":"2024-09-17T21:49:25.726819+0000","last_fullsized":"2024-09-17T21:49:25.726819+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:41:59.923845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046368620000000003,"stat_sum":{"num_bytes":26521160,"num_objects":67,"num_object_clones":10,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":13954,"num_read_kb":73967,"num_write":984,"num_write_kb":17155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.15","version":"206'673","reported_seq":9612,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.685647+0000","last_change":"2024-09-17T21:49:28.685647+0000","last_active":"2024-09-17T21:49:28.685647+0000","last_peered":"2024-09-17T21:49:28.685647+0000","last_clean":"2024-09-17T21:49:28.685647+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:49:28.685647+0000","last_undegraded":"2024-09-17T21:49:28.685647+0000","last_fullsized":"2024-09-17T21:49:28.685647+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:55:12.749791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059210449999999998,"stat_sum":{"num_bytes":29674019,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":29,"num_read":16399,"num_read_kb":59360,"num_write":657,"num_write_kb":14183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.14","version":"193'451","reported_seq":18962,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.717684+0000","last_change":"2024-09-17T21:49:24.718517+0000","last_active":"2024-09-17T21:49:24.717684+0000","last_peered":"2024-09-17T21:49:24.717684+0000","last_clean":"2024-09-17T21:49:24.717684+0000","last_became_active":"2024-09-17T21:47:17.915249+0000","last_became_peered":"2024-09-17T21:47:17.915249+0000","last_unstale":"2024-09-17T21:49:24.717684+0000","last_undegraded":"2024-09-17T21:49:24.717684+0000","last_fullsized":"2024-09-17T21:49:24.717684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:49.538046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017207959999999999,"stat_sum":{"num_bytes":93252716,"num_objects":69,"num_object_clones":16,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":18066,"num_read_kb":143498,"num_write":528,"num_write_kb":43999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":39957736,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679677+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:49:28.679677+0000","last_peered":"2024-09-17T21:49:28.679677+0000","last_clean":"2024-09-17T21:49:28.679677+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:49:28.679677+0000","last_undegraded":"2024-09-17T21:49:28.679677+0000","last_fullsized":"2024-09-17T21:49:28.679677+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"203'664","reported_seq":10218,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.818794+0000","last_change":"2024-09-17T21:49:25.726348+0000","last_active":"2024-09-17T21:49:25.818794+0000","last_peered":"2024-09-17T21:49:25.818794+0000","last_clean":"2024-09-17T21:49:25.818794+0000","last_became_active":"2024-09-17T21:48:10.691607+0000","last_became_peered":"2024-09-17T21:48:10.691607+0000","last_unstale":"2024-09-17T21:49:25.818794+0000","last_undegraded":"2024-09-17T21:49:25.818794+0000","last_fullsized":"2024-09-17T21:49:25.818794+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:15.519317+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0075190309999999998,"stat_sum":{"num_bytes":39017664,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":12091,"num_read_kb":53809,"num_write":572,"num_write_kb":16793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":45049091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.6","version":"0'0","reported_seq":421,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670075+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:49:28.670075+0000","last_peered":"2024-09-17T21:49:28.670075+0000","last_clean":"2024-09-17T21:49:28.670075+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:49:28.670075+0000","last_undegraded":"2024-09-17T21:49:28.670075+0000","last_fullsized":"2024-09-17T21:49:28.670075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"202'514","reported_seq":6561,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.719509+0000","last_change":"2024-09-17T21:49:25.720139+0000","last_active":"2024-09-17T21:49:25.719509+0000","last_peered":"2024-09-17T21:49:25.719509+0000","last_clean":"2024-09-17T21:49:25.719509+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:49:25.719509+0000","last_undegraded":"2024-09-17T21:49:25.719509+0000","last_fullsized":"2024-09-17T21:49:25.719509+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:32.403016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178634,"stat_sum":{"num_bytes":44294739,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":29,"num_read":6389,"num_read_kb":43090,"num_write":614,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679904+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:49:28.679904+0000","last_peered":"2024-09-17T21:49:28.679904+0000","last_clean":"2024-09-17T21:49:28.679904+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:49:28.679904+0000","last_undegraded":"2024-09-17T21:49:28.679904+0000","last_fullsized":"2024-09-17T21:49:28.679904+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"205'611","reported_seq":18946,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.804966+0000","last_change":"2024-09-17T21:49:28.671730+0000","last_active":"2024-09-17T21:49:28.804966+0000","last_peered":"2024-09-17T21:49:28.804966+0000","last_clean":"2024-09-17T21:49:28.804966+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:49:28.804966+0000","last_undegraded":"2024-09-17T21:49:28.804966+0000","last_fullsized":"2024-09-17T21:49:28.804966+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:46.423417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015513980000000001,"stat_sum":{"num_bytes":73134080,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":18493,"num_read_kb":88075,"num_write":443,"num_write_kb":12125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":429,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670774+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:49:28.670774+0000","last_peered":"2024-09-17T21:49:28.670774+0000","last_clean":"2024-09-17T21:49:28.670774+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:49:28.670774+0000","last_undegraded":"2024-09-17T21:49:28.670774+0000","last_fullsized":"2024-09-17T21:49:28.670774+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'474","reported_seq":12654,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726459+0000","last_change":"2024-09-17T21:49:25.726729+0000","last_active":"2024-09-17T21:49:25.726459+0000","last_peered":"2024-09-17T21:49:25.726459+0000","last_clean":"2024-09-17T21:49:25.726459+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:49:25.726459+0000","last_undegraded":"2024-09-17T21:49:25.726459+0000","last_fullsized":"2024-09-17T21:49:25.726459+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:44.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004370311,"stat_sum":{"num_bytes":35029084,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":11374,"num_read_kb":39459,"num_write":476,"num_write_kb":17805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.3","version":"202'799","reported_seq":11197,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.825839+0000","last_change":"2024-09-17T21:49:24.717637+0000","last_active":"2024-09-17T21:49:24.825839+0000","last_peered":"2024-09-17T21:49:24.825839+0000","last_clean":"2024-09-17T21:49:24.825839+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:49:24.825839+0000","last_undegraded":"2024-09-17T21:49:24.825839+0000","last_fullsized":"2024-09-17T21:49:24.825839+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:06.970137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091330100000000004,"stat_sum":{"num_bytes":31222723,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":13751,"num_read_kb":38349,"num_write":1045,"num_write_kb":10549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"2.2","version":"19'2","reported_seq":390,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.721988+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:49:25.721988+0000","last_peered":"2024-09-17T21:49:25.721988+0000","last_clean":"2024-09-17T21:49:25.721988+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:49:25.721988+0000","last_undegraded":"2024-09-17T21:49:25.721988+0000","last_fullsized":"2024-09-17T21:49:25.721988+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"206'425","reported_seq":7078,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670924+0000","last_change":"2024-09-17T21:49:28.670924+0000","last_active":"2024-09-17T21:49:28.670924+0000","last_peered":"2024-09-17T21:49:28.670924+0000","last_clean":"2024-09-17T21:49:28.670924+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:49:28.670924+0000","last_undegraded":"2024-09-17T21:49:28.670924+0000","last_fullsized":"2024-09-17T21:49:28.670924+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:19.268701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001607198,"stat_sum":{"num_bytes":24009851,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":10028,"num_read_kb":30205,"num_write":346,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.718648+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:49:25.718648+0000","last_peered":"2024-09-17T21:49:25.718648+0000","last_clean":"2024-09-17T21:49:25.718648+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:49:25.718648+0000","last_undegraded":"2024-09-17T21:49:25.718648+0000","last_fullsized":"2024-09-17T21:49:25.718648+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'1748","reported_seq":8564,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.723129+0000","last_change":"2024-09-17T21:49:25.723803+0000","last_active":"2024-09-17T21:49:25.723129+0000","last_peered":"2024-09-17T21:49:25.723129+0000","last_clean":"2024-09-17T21:49:25.723129+0000","last_became_active":"2024-09-17T21:48:10.690220+0000","last_became_peered":"2024-09-17T21:48:10.690220+0000","last_unstale":"2024-09-17T21:49:25.723129+0000","last_undegraded":"2024-09-17T21:49:25.723129+0000","last_fullsized":"2024-09-17T21:49:25.723129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1748,"log_dups_size":0,"ondisk_log_size":1748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:03.213184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047717369999999999,"stat_sum":{"num_bytes":45540692,"num_objects":67,"num_object_clones":10,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":33,"num_read":14997,"num_read_kb":56346,"num_write":2655,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679369+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:49:28.679369+0000","last_peered":"2024-09-17T21:49:28.679369+0000","last_clean":"2024-09-17T21:49:28.679369+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:49:28.679369+0000","last_undegraded":"2024-09-17T21:49:28.679369+0000","last_fullsized":"2024-09-17T21:49:28.679369+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"206'469","reported_seq":5431,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.671802+0000","last_change":"2024-09-17T21:49:28.671802+0000","last_active":"2024-09-17T21:49:28.671802+0000","last_peered":"2024-09-17T21:49:28.671802+0000","last_clean":"2024-09-17T21:49:28.671802+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:49:28.671802+0000","last_undegraded":"2024-09-17T21:49:28.671802+0000","last_fullsized":"2024-09-17T21:49:28.671802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:23.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001285734,"stat_sum":{"num_bytes":35655230,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":5754,"num_read_kb":32079,"num_write":531,"num_write_kb":18464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.3","version":"17'1","reported_seq":407,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722099+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:49:25.722099+0000","last_peered":"2024-09-17T21:49:25.722099+0000","last_clean":"2024-09-17T21:49:25.722099+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:49:25.722099+0000","last_undegraded":"2024-09-17T21:49:25.722099+0000","last_fullsized":"2024-09-17T21:49:25.722099+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":542,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679379+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:49:28.679379+0000","last_peered":"2024-09-17T21:49:28.679379+0000","last_clean":"2024-09-17T21:49:28.679379+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:49:28.679379+0000","last_undegraded":"2024-09-17T21:49:28.679379+0000","last_fullsized":"2024-09-17T21:49:28.679379+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"199'430","reported_seq":7007,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.720227+0000","last_change":"2024-09-17T21:49:25.721266+0000","last_active":"2024-09-17T21:49:25.720227+0000","last_peered":"2024-09-17T21:49:25.720227+0000","last_clean":"2024-09-17T21:49:25.720227+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:49:25.720227+0000","last_undegraded":"2024-09-17T21:49:25.720227+0000","last_fullsized":"2024-09-17T21:49:25.720227+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:32.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020511639999999999,"stat_sum":{"num_bytes":28745849,"num_objects":53,"num_object_clones":6,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":7233,"num_read_kb":58585,"num_write":487,"num_write_kb":6067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.9","version":"199'746","reported_seq":10627,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.843623+0000","last_change":"2024-09-17T21:49:24.718281+0000","last_active":"2024-09-17T21:49:24.843623+0000","last_peered":"2024-09-17T21:49:24.843623+0000","last_clean":"2024-09-17T21:49:24.843623+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:49:24.843623+0000","last_undegraded":"2024-09-17T21:49:24.843623+0000","last_fullsized":"2024-09-17T21:49:24.843623+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:20.156726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013311429999999999,"stat_sum":{"num_bytes":59553481,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":15348,"num_read_kb":69972,"num_write":717,"num_write_kb":25447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"3.a","version":"206'490","reported_seq":5110,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.685362+0000","last_change":"2024-09-17T21:49:28.685362+0000","last_active":"2024-09-17T21:49:28.685362+0000","last_peered":"2024-09-17T21:49:28.685362+0000","last_clean":"2024-09-17T21:49:28.685362+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:49:28.685362+0000","last_undegraded":"2024-09-17T21:49:28.685362+0000","last_fullsized":"2024-09-17T21:49:28.685362+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:53.608463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057057109999999996,"stat_sum":{"num_bytes":7285376,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":4858,"num_read_kb":14254,"num_write":519,"num_write_kb":2537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.b","version":"206'506","reported_seq":10537,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.765193+0000","last_change":"2024-09-17T21:49:28.670938+0000","last_active":"2024-09-17T21:49:28.765193+0000","last_peered":"2024-09-17T21:49:28.765193+0000","last_clean":"2024-09-17T21:49:28.765193+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:49:28.765193+0000","last_undegraded":"2024-09-17T21:49:28.765193+0000","last_fullsized":"2024-09-17T21:49:28.765193+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:37.201931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011557589999999999,"stat_sum":{"num_bytes":32895564,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":27,"num_read":10495,"num_read_kb":57866,"num_write":487,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.c","version":"199'537","reported_seq":18938,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726038+0000","last_change":"2024-09-17T21:49:25.726385+0000","last_active":"2024-09-17T21:49:25.726038+0000","last_peered":"2024-09-17T21:49:25.726038+0000","last_clean":"2024-09-17T21:49:25.726038+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:49:25.726038+0000","last_undegraded":"2024-09-17T21:49:25.726038+0000","last_fullsized":"2024-09-17T21:49:25.726038+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:16.981796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040170470000000002,"stat_sum":{"num_bytes":39486913,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":18022,"num_read_kb":76883,"num_write":551,"num_write_kb":16858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.d","version":"206'775","reported_seq":9670,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.688090+0000","last_change":"2024-09-17T21:49:28.688090+0000","last_active":"2024-09-17T21:49:28.688090+0000","last_peered":"2024-09-17T21:49:28.688090+0000","last_clean":"2024-09-17T21:49:28.688090+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:49:28.688090+0000","last_undegraded":"2024-09-17T21:49:28.688090+0000","last_fullsized":"2024-09-17T21:49:28.688090+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:54.341771+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0079892009999999996,"stat_sum":{"num_bytes":115901535,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":11342,"num_read_kb":74018,"num_write":879,"num_write_kb":32170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.e","version":"195'398","reported_seq":5807,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.727213+0000","last_change":"2024-09-17T21:49:25.727473+0000","last_active":"2024-09-17T21:49:25.727213+0000","last_peered":"2024-09-17T21:49:25.727213+0000","last_clean":"2024-09-17T21:49:25.727213+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:49:25.727213+0000","last_undegraded":"2024-09-17T21:49:25.727213+0000","last_fullsized":"2024-09-17T21:49:25.727213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:14.569434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049814560000000004,"stat_sum":{"num_bytes":14734982,"num_objects":51,"num_object_clones":8,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":31,"num_read":5587,"num_read_kb":34401,"num_write":572,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.f","version":"203'921","reported_seq":17163,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.812705+0000","last_change":"2024-09-17T21:49:25.723902+0000","last_active":"2024-09-17T21:49:25.812705+0000","last_peered":"2024-09-17T21:49:25.812705+0000","last_clean":"2024-09-17T21:49:25.812705+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:49:25.812705+0000","last_undegraded":"2024-09-17T21:49:25.812705+0000","last_fullsized":"2024-09-17T21:49:25.812705+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:22.457056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016790150000000001,"stat_sum":{"num_bytes":37204697,"num_objects":62,"num_object_clones":9,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":19012,"num_read_kb":65969,"num_write":1276,"num_write_kb":30218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.10","version":"196'536","reported_seq":6871,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722508+0000","last_change":"2024-09-17T21:49:25.722965+0000","last_active":"2024-09-17T21:49:25.722508+0000","last_peered":"2024-09-17T21:49:25.722508+0000","last_clean":"2024-09-17T21:49:25.722508+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:49:25.722508+0000","last_undegraded":"2024-09-17T21:49:25.722508+0000","last_fullsized":"2024-09-17T21:49:25.722508+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:05.202832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091281399999999996,"stat_sum":{"num_bytes":35577151,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":8170,"num_read_kb":24168,"num_write":572,"num_write_kb":5419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.11","version":"206'499","reported_seq":6110,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.684299+0000","last_change":"2024-09-17T21:49:28.684868+0000","last_active":"2024-09-17T21:49:28.684299+0000","last_peered":"2024-09-17T21:49:28.684299+0000","last_clean":"2024-09-17T21:49:28.684299+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:49:28.684299+0000","last_undegraded":"2024-09-17T21:49:28.684299+0000","last_fullsized":"2024-09-17T21:49:28.684299+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:45:54.549048+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0052626970000000002,"stat_sum":{"num_bytes":88639024,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":34,"num_read":4764,"num_read_kb":61218,"num_write":566,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.12","version":"203'360","reported_seq":12418,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722010+0000","last_change":"2024-09-17T21:49:25.722360+0000","last_active":"2024-09-17T21:49:25.722010+0000","last_peered":"2024-09-17T21:49:25.722010+0000","last_clean":"2024-09-17T21:49:25.722010+0000","last_became_active":"2024-09-17T21:49:22.711125+0000","last_became_peered":"2024-09-17T21:49:22.711125+0000","last_unstale":"2024-09-17T21:49:25.722010+0000","last_undegraded":"2024-09-17T21:49:25.722010+0000","last_fullsized":"2024-09-17T21:49:25.722010+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:00.273705+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037584490000000001,"stat_sum":{"num_bytes":27157580,"num_objects":42,"num_object_clones":3,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":10136,"num_read_kb":51965,"num_write":441,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":54272456,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.13","version":"206'443","reported_seq":10126,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.684137+0000","last_change":"2024-09-17T21:49:28.684455+0000","last_active":"2024-09-17T21:49:28.684137+0000","last_peered":"2024-09-17T21:49:28.684137+0000","last_clean":"2024-09-17T21:49:28.684137+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:49:28.684137+0000","last_undegraded":"2024-09-17T21:49:28.684137+0000","last_fullsized":"2024-09-17T21:49:28.684137+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:37.178775+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0048708349999999996,"stat_sum":{"num_bytes":38477562,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":8718,"num_read_kb":42428,"num_write":549,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1565486200,"num_objects":1840,"num_object_clones":295,"num_object_copies":3680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1840,"num_whiteouts":894,"num_read":371479,"num_read_kb":1867832,"num_write":20919,"num_write_kb":601854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10848,"num_bytes_recovered":1229762475,"num_keys_recovered":2095,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675573760,"data_stored":984212955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":18623,"ondisk_log_size":18623,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607600,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":197240,"kb_used_data":143832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94174600,"statfs":{"total":96636764160,"available":96434790400,"internally_reserved":0,"allocated":147283968,"data_stored":228466396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159821,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171652,"kb_used_data":133988,"kb_used_omap":28,"kb_used_meta":37283,"kb_avail":94200188,"statfs":{"total":96636764160,"available":96460992512,"internally_reserved":0,"allocated":137203712,"data_stored":200431941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":38177872},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607599,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147716,"kb_used_data":98472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94224124,"statfs":{"total":96636764160,"available":96485502976,"internally_reserved":0,"allocated":100835328,"data_stored":149272677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141812,"kb_used_data":88416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94230028,"statfs":{"total":96636764160,"available":96491548672,"internally_reserved":0,"allocated":90537984,"data_stored":108102663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607599,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219912,"kb_used_data":144308,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94151928,"statfs":{"total":96636764160,"available":96411574272,"internally_reserved":0,"allocated":147771392,"data_stored":213430538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125868,"kb_used_data":72472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94245972,"statfs":{"total":96636764160,"available":96507875328,"internally_reserved":0,"allocated":74211328,"data_stored":98265091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87502848,"data_stored":106482534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67534848,"data_stored":93020495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98390016,"data_stored":148239844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131989504,"data_stored":196649604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":145317888,"data_stored":212383862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":144838656,"data_stored":227436616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:49:30.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.149480073999996, 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:30.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:30.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:30.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148321741000004, 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:30.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:30.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148305195999995, 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:30.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148290431000007, 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148263126000003, 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148226238000007, 2024-09-17T21:49:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148187539999995, 2024-09-17T21:49:30.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:30.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:30.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:30.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.148149367000002, 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.271 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.275 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.147861027000005, 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.276 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:30.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:30.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 92.147834431000007, 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:30.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.573296624999998, 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:30.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:30.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:30.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.573263310000002, 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:30.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:30.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.566173246000005, 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:30.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:30.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.566139566000004, 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:30.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:30.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.308 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.477670635999999, 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:30.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:30.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:30.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:30.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.477577617000001, 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:30.318 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:30.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.324 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.474947669999999, 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:30.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:30.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:30.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:30.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 60.131404967000002, 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:30.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:30.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.476085470000001, 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.339 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:30.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:30.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 59.470176440000003, 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.345 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:30.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.351 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:30.353 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:30.350+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:30.453 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:30.451+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:30.484 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:30.485 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:49:30.497 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-17T21:49:30.137610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1566076587,"num_objects":1844,"num_object_clones":295,"num_object_copies":3688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1844,"num_whiteouts":894,"num_read":371595,"num_read_kb":1867932,"num_write":21080,"num_write_kb":604272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10866,"num_bytes_recovered":1234485419,"num_keys_recovered":2095,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18743,"ondisk_log_size":18743,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":1004200,"kb_used_data":681488,"kb_used_omap":36,"kb_used_meta":322203,"kb_avail":565226840,"statfs":{"total":579820584960,"available":578792284160,"internally_reserved":0,"allocated":697843712,"data_stored":997969306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":329936169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-77031375,"num_objects":-109,"num_object_clones":-37,"num_object_copies":-218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-513,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-109,"num_whiteouts":-32,"num_read":567,"num_read_kb":478,"num_write":117,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":69395503,"num_keys_recovered":95,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000683"},"pg_stats":[{"pgid":"3.1f","version":"202'353","reported_seq":11463,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722437+0000","last_change":"2024-09-17T21:49:25.723051+0000","last_active":"2024-09-17T21:49:25.722437+0000","last_peered":"2024-09-17T21:49:25.722437+0000","last_clean":"2024-09-17T21:49:25.722437+0000","last_became_active":"2024-09-17T21:49:23.715014+0000","last_became_peered":"2024-09-17T21:49:23.715014+0000","last_unstale":"2024-09-17T21:49:25.722437+0000","last_undegraded":"2024-09-17T21:49:25.722437+0000","last_fullsized":"2024-09-17T21:49:25.722437+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:39:45.314185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040224290000000001,"stat_sum":{"num_bytes":51420875,"num_objects":67,"num_object_clones":7,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":30,"num_read":11121,"num_read_kb":70761,"num_write":333,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1e","version":"203'512","reported_seq":20104,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726370+0000","last_change":"2024-09-17T21:49:25.726370+0000","last_active":"2024-09-17T21:49:25.726370+0000","last_peered":"2024-09-17T21:49:25.726370+0000","last_clean":"2024-09-17T21:49:25.726370+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:49:25.726370+0000","last_undegraded":"2024-09-17T21:49:25.726370+0000","last_fullsized":"2024-09-17T21:49:25.726370+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:22:03.506844+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0069220180000000003,"stat_sum":{"num_bytes":106936864,"num_objects":74,"num_object_clones":17,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":34,"num_read":18711,"num_read_kb":109023,"num_write":400,"num_write_kb":34487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1d","version":"203'377","reported_seq":5221,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.725277+0000","last_change":"2024-09-17T21:49:25.725740+0000","last_active":"2024-09-17T21:49:25.725277+0000","last_peered":"2024-09-17T21:49:25.725277+0000","last_clean":"2024-09-17T21:49:25.725277+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:49:25.725277+0000","last_undegraded":"2024-09-17T21:49:25.725277+0000","last_fullsized":"2024-09-17T21:49:25.725277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":377,"log_dups_size":0,"ondisk_log_size":377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:05:33.608857+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0035601869999999998,"stat_sum":{"num_bytes":34531401,"num_objects":52,"num_object_clones":7,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":31,"num_read":4895,"num_read_kb":51120,"num_write":639,"num_write_kb":31818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1c","version":"203'558","reported_seq":18089,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.759849+0000","last_change":"2024-09-17T21:49:25.727050+0000","last_active":"2024-09-17T21:49:25.759849+0000","last_peered":"2024-09-17T21:49:25.759849+0000","last_clean":"2024-09-17T21:49:25.759849+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:49:25.759849+0000","last_undegraded":"2024-09-17T21:49:25.759849+0000","last_fullsized":"2024-09-17T21:49:25.759849+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:38:28.084682+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.004765895,"stat_sum":{"num_bytes":74653797,"num_objects":65,"num_object_clones":14,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":26,"num_read":19621,"num_read_kb":86798,"num_write":606,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1b","version":"202'426","reported_seq":16696,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.720426+0000","last_change":"2024-09-17T21:49:25.720426+0000","last_active":"2024-09-17T21:49:25.720426+0000","last_peered":"2024-09-17T21:49:25.720426+0000","last_clean":"2024-09-17T21:49:25.720426+0000","last_became_active":"2024-09-17T21:48:08.675352+0000","last_became_peered":"2024-09-17T21:48:08.675352+0000","last_unstale":"2024-09-17T21:49:25.720426+0000","last_undegraded":"2024-09-17T21:49:25.720426+0000","last_fullsized":"2024-09-17T21:49:25.720426+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:18:02.740360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00165318,"stat_sum":{"num_bytes":66587664,"num_objects":53,"num_object_clones":10,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":21,"num_read":14405,"num_read_kb":43490,"num_write":345,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.1a","version":"202'623","reported_seq":11331,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.842681+0000","last_change":"2024-09-17T21:49:24.717627+0000","last_active":"2024-09-17T21:49:24.842681+0000","last_peered":"2024-09-17T21:49:24.842681+0000","last_clean":"2024-09-17T21:49:24.842681+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:49:24.842681+0000","last_undegraded":"2024-09-17T21:49:24.842681+0000","last_fullsized":"2024-09-17T21:49:24.842681+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:44:46.986734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010175710000000001,"stat_sum":{"num_bytes":98333436,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":13366,"num_read_kb":81957,"num_write":660,"num_write_kb":25541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"3.19","version":"206'397","reported_seq":4818,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.764000+0000","last_change":"2024-09-17T21:49:28.672321+0000","last_active":"2024-09-17T21:49:28.764000+0000","last_peered":"2024-09-17T21:49:28.764000+0000","last_clean":"2024-09-17T21:49:28.764000+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:49:28.764000+0000","last_undegraded":"2024-09-17T21:49:28.764000+0000","last_fullsized":"2024-09-17T21:49:28.764000+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:01:54.013206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001666739,"stat_sum":{"num_bytes":33117314,"num_objects":52,"num_object_clones":6,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":29,"num_read":4426,"num_read_kb":33920,"num_write":402,"num_write_kb":13444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.18","version":"203'616","reported_seq":12855,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.721818+0000","last_change":"2024-09-17T21:49:25.722167+0000","last_active":"2024-09-17T21:49:25.721818+0000","last_peered":"2024-09-17T21:49:25.721818+0000","last_clean":"2024-09-17T21:49:25.721818+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:49:25.721818+0000","last_undegraded":"2024-09-17T21:49:25.721818+0000","last_fullsized":"2024-09-17T21:49:25.721818+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:33:02.344566+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003434327,"stat_sum":{"num_bytes":77495033,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":29,"num_read":12050,"num_read_kb":68756,"num_write":731,"num_write_kb":13790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.17","version":"196'358","reported_seq":7544,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.719438+0000","last_change":"2024-09-17T21:49:25.720393+0000","last_active":"2024-09-17T21:49:25.719438+0000","last_peered":"2024-09-17T21:49:25.719438+0000","last_clean":"2024-09-17T21:49:25.719438+0000","last_became_active":"2024-09-17T21:48:10.691486+0000","last_became_peered":"2024-09-17T21:48:10.691486+0000","last_unstale":"2024-09-17T21:49:25.719438+0000","last_undegraded":"2024-09-17T21:49:25.719438+0000","last_fullsized":"2024-09-17T21:49:25.719438+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:41.918512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012261640000000001,"stat_sum":{"num_bytes":19398144,"num_objects":50,"num_object_clones":7,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":30,"num_read":7903,"num_read_kb":32043,"num_write":336,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.16","version":"202'934","reported_seq":11533,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726819+0000","last_change":"2024-09-17T21:49:25.727071+0000","last_active":"2024-09-17T21:49:25.726819+0000","last_peered":"2024-09-17T21:49:25.726819+0000","last_clean":"2024-09-17T21:49:25.726819+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:49:25.726819+0000","last_undegraded":"2024-09-17T21:49:25.726819+0000","last_fullsized":"2024-09-17T21:49:25.726819+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":934,"log_dups_size":0,"ondisk_log_size":934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:41:59.923845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0046368620000000003,"stat_sum":{"num_bytes":26521160,"num_objects":67,"num_object_clones":10,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":40,"num_read":13954,"num_read_kb":73967,"num_write":984,"num_write_kb":17155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.15","version":"206'673","reported_seq":9612,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.685647+0000","last_change":"2024-09-17T21:49:28.685647+0000","last_active":"2024-09-17T21:49:28.685647+0000","last_peered":"2024-09-17T21:49:28.685647+0000","last_clean":"2024-09-17T21:49:28.685647+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:49:28.685647+0000","last_undegraded":"2024-09-17T21:49:28.685647+0000","last_fullsized":"2024-09-17T21:49:28.685647+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":673,"log_dups_size":0,"ondisk_log_size":673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:55:12.749791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059210449999999998,"stat_sum":{"num_bytes":29674019,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":29,"num_read":16399,"num_read_kb":59360,"num_write":657,"num_write_kb":14183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.14","version":"193'451","reported_seq":18962,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.717684+0000","last_change":"2024-09-17T21:49:24.718517+0000","last_active":"2024-09-17T21:49:24.717684+0000","last_peered":"2024-09-17T21:49:24.717684+0000","last_clean":"2024-09-17T21:49:24.717684+0000","last_became_active":"2024-09-17T21:47:17.915249+0000","last_became_peered":"2024-09-17T21:47:17.915249+0000","last_unstale":"2024-09-17T21:49:24.717684+0000","last_undegraded":"2024-09-17T21:49:24.717684+0000","last_fullsized":"2024-09-17T21:49:24.717684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":451,"log_dups_size":0,"ondisk_log_size":451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:01:49.538046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017207959999999999,"stat_sum":{"num_bytes":93252716,"num_objects":69,"num_object_clones":16,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":26,"num_read":18066,"num_read_kb":143498,"num_write":528,"num_write_kb":43999,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":39957736,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"2.7","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679677+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:49:28.679677+0000","last_peered":"2024-09-17T21:49:28.679677+0000","last_clean":"2024-09-17T21:49:28.679677+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:49:28.679677+0000","last_undegraded":"2024-09-17T21:49:28.679677+0000","last_fullsized":"2024-09-17T21:49:28.679677+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"203'664","reported_seq":10218,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.818794+0000","last_change":"2024-09-17T21:49:25.726348+0000","last_active":"2024-09-17T21:49:25.818794+0000","last_peered":"2024-09-17T21:49:25.818794+0000","last_clean":"2024-09-17T21:49:25.818794+0000","last_became_active":"2024-09-17T21:48:10.691607+0000","last_became_peered":"2024-09-17T21:48:10.691607+0000","last_unstale":"2024-09-17T21:49:25.818794+0000","last_undegraded":"2024-09-17T21:49:25.818794+0000","last_fullsized":"2024-09-17T21:49:25.818794+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":664,"log_dups_size":0,"ondisk_log_size":664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:30:15.519317+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0075190309999999998,"stat_sum":{"num_bytes":39017664,"num_objects":58,"num_object_clones":9,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":30,"num_read":12091,"num_read_kb":53809,"num_write":572,"num_write_kb":16793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":45049091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.6","version":"0'0","reported_seq":421,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670075+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:49:28.670075+0000","last_peered":"2024-09-17T21:49:28.670075+0000","last_clean":"2024-09-17T21:49:28.670075+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:49:28.670075+0000","last_undegraded":"2024-09-17T21:49:28.670075+0000","last_fullsized":"2024-09-17T21:49:28.670075+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"202'514","reported_seq":6561,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.719509+0000","last_change":"2024-09-17T21:49:25.720139+0000","last_active":"2024-09-17T21:49:25.719509+0000","last_peered":"2024-09-17T21:49:25.719509+0000","last_clean":"2024-09-17T21:49:25.719509+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:49:25.719509+0000","last_undegraded":"2024-09-17T21:49:25.719509+0000","last_fullsized":"2024-09-17T21:49:25.719509+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:32.403016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178634,"stat_sum":{"num_bytes":44294739,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":29,"num_read":6389,"num_read_kb":43090,"num_write":614,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.5","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679904+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:49:28.679904+0000","last_peered":"2024-09-17T21:49:28.679904+0000","last_clean":"2024-09-17T21:49:28.679904+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:49:28.679904+0000","last_undegraded":"2024-09-17T21:49:28.679904+0000","last_fullsized":"2024-09-17T21:49:28.679904+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"205'611","reported_seq":18946,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.804966+0000","last_change":"2024-09-17T21:49:28.671730+0000","last_active":"2024-09-17T21:49:28.804966+0000","last_peered":"2024-09-17T21:49:28.804966+0000","last_clean":"2024-09-17T21:49:28.804966+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:49:28.804966+0000","last_undegraded":"2024-09-17T21:49:28.804966+0000","last_fullsized":"2024-09-17T21:49:28.804966+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:46.423417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015513980000000001,"stat_sum":{"num_bytes":73134080,"num_objects":52,"num_object_clones":8,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":26,"num_read":18493,"num_read_kb":88075,"num_write":443,"num_write_kb":12125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":429,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670774+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:49:28.670774+0000","last_peered":"2024-09-17T21:49:28.670774+0000","last_clean":"2024-09-17T21:49:28.670774+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:49:28.670774+0000","last_undegraded":"2024-09-17T21:49:28.670774+0000","last_fullsized":"2024-09-17T21:49:28.670774+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"202'474","reported_seq":12654,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726459+0000","last_change":"2024-09-17T21:49:25.726729+0000","last_active":"2024-09-17T21:49:25.726459+0000","last_peered":"2024-09-17T21:49:25.726459+0000","last_clean":"2024-09-17T21:49:25.726459+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:49:25.726459+0000","last_undegraded":"2024-09-17T21:49:25.726459+0000","last_fullsized":"2024-09-17T21:49:25.726459+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:50:44.696653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004370311,"stat_sum":{"num_bytes":35029084,"num_objects":62,"num_object_clones":8,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":28,"num_read":11374,"num_read_kb":39459,"num_write":476,"num_write_kb":17805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.3","version":"202'799","reported_seq":11197,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.825839+0000","last_change":"2024-09-17T21:49:24.717637+0000","last_active":"2024-09-17T21:49:24.825839+0000","last_peered":"2024-09-17T21:49:24.825839+0000","last_clean":"2024-09-17T21:49:24.825839+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:49:24.825839+0000","last_undegraded":"2024-09-17T21:49:24.825839+0000","last_fullsized":"2024-09-17T21:49:24.825839+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:06.970137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091330100000000004,"stat_sum":{"num_bytes":31222723,"num_objects":54,"num_object_clones":8,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":21,"num_read":13751,"num_read_kb":38349,"num_write":1045,"num_write_kb":10549,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"2.2","version":"19'2","reported_seq":390,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.721988+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:49:25.721988+0000","last_peered":"2024-09-17T21:49:25.721988+0000","last_clean":"2024-09-17T21:49:25.721988+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:49:25.721988+0000","last_undegraded":"2024-09-17T21:49:25.721988+0000","last_fullsized":"2024-09-17T21:49:25.721988+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"206'425","reported_seq":7078,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.670924+0000","last_change":"2024-09-17T21:49:28.670924+0000","last_active":"2024-09-17T21:49:28.670924+0000","last_peered":"2024-09-17T21:49:28.670924+0000","last_clean":"2024-09-17T21:49:28.670924+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:49:28.670924+0000","last_undegraded":"2024-09-17T21:49:28.670924+0000","last_fullsized":"2024-09-17T21:49:28.670924+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:52:19.268701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001607198,"stat_sum":{"num_bytes":24009851,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":30,"num_read":10028,"num_read_kb":30205,"num_write":346,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.718648+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:49:25.718648+0000","last_peered":"2024-09-17T21:49:25.718648+0000","last_clean":"2024-09-17T21:49:25.718648+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:49:25.718648+0000","last_undegraded":"2024-09-17T21:49:25.718648+0000","last_fullsized":"2024-09-17T21:49:25.718648+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"202'1748","reported_seq":8564,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.723129+0000","last_change":"2024-09-17T21:49:25.723803+0000","last_active":"2024-09-17T21:49:25.723129+0000","last_peered":"2024-09-17T21:49:25.723129+0000","last_clean":"2024-09-17T21:49:25.723129+0000","last_became_active":"2024-09-17T21:48:10.690220+0000","last_became_peered":"2024-09-17T21:48:10.690220+0000","last_unstale":"2024-09-17T21:49:25.723129+0000","last_undegraded":"2024-09-17T21:49:25.723129+0000","last_fullsized":"2024-09-17T21:49:25.723129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1748,"log_dups_size":0,"ondisk_log_size":1748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:16:03.213184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0047717369999999999,"stat_sum":{"num_bytes":45540692,"num_objects":67,"num_object_clones":10,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":33,"num_read":14997,"num_read_kb":56346,"num_write":2655,"num_write_kb":12543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"2.0","version":"0'0","reported_seq":28,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679369+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:49:28.679369+0000","last_peered":"2024-09-17T21:49:28.679369+0000","last_clean":"2024-09-17T21:49:28.679369+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:49:28.679369+0000","last_undegraded":"2024-09-17T21:49:28.679369+0000","last_fullsized":"2024-09-17T21:49:28.679369+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"206'469","reported_seq":5431,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.671802+0000","last_change":"2024-09-17T21:49:28.671802+0000","last_active":"2024-09-17T21:49:28.671802+0000","last_peered":"2024-09-17T21:49:28.671802+0000","last_clean":"2024-09-17T21:49:28.671802+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:49:28.671802+0000","last_undegraded":"2024-09-17T21:49:28.671802+0000","last_fullsized":"2024-09-17T21:49:28.671802+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:25:23.941159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001285734,"stat_sum":{"num_bytes":35655230,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":25,"num_read":5754,"num_read_kb":32079,"num_write":531,"num_write_kb":18464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"2.3","version":"17'1","reported_seq":407,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722099+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:49:25.722099+0000","last_peered":"2024-09-17T21:49:25.722099+0000","last_clean":"2024-09-17T21:49:25.722099+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:49:25.722099+0000","last_undegraded":"2024-09-17T21:49:25.722099+0000","last_fullsized":"2024-09-17T21:49:25.722099+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":542,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.679379+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:49:28.679379+0000","last_peered":"2024-09-17T21:49:28.679379+0000","last_clean":"2024-09-17T21:49:28.679379+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:49:28.679379+0000","last_undegraded":"2024-09-17T21:49:28.679379+0000","last_fullsized":"2024-09-17T21:49:28.679379+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"199'430","reported_seq":7007,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.720227+0000","last_change":"2024-09-17T21:49:25.721266+0000","last_active":"2024-09-17T21:49:25.720227+0000","last_peered":"2024-09-17T21:49:25.720227+0000","last_clean":"2024-09-17T21:49:25.720227+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:49:25.720227+0000","last_undegraded":"2024-09-17T21:49:25.720227+0000","last_fullsized":"2024-09-17T21:49:25.720227+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:32.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020511639999999999,"stat_sum":{"num_bytes":28745849,"num_objects":53,"num_object_clones":6,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":27,"num_read":7233,"num_read_kb":58585,"num_write":487,"num_write_kb":6067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.9","version":"199'746","reported_seq":10627,"reported_epoch":202,"state":"active+clean","last_fresh":"2024-09-17T21:49:24.843623+0000","last_change":"2024-09-17T21:49:24.718281+0000","last_active":"2024-09-17T21:49:24.843623+0000","last_peered":"2024-09-17T21:49:24.843623+0000","last_clean":"2024-09-17T21:49:24.843623+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:49:24.843623+0000","last_undegraded":"2024-09-17T21:49:24.843623+0000","last_fullsized":"2024-09-17T21:49:24.843623+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:10:20.156726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013311429999999999,"stat_sum":{"num_bytes":59553481,"num_objects":55,"num_object_clones":10,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":15348,"num_read_kb":69972,"num_write":717,"num_write_kb":25447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2b","length":"b"}]},{"pgid":"3.a","version":"206'490","reported_seq":5110,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.685362+0000","last_change":"2024-09-17T21:49:28.685362+0000","last_active":"2024-09-17T21:49:28.685362+0000","last_peered":"2024-09-17T21:49:28.685362+0000","last_clean":"2024-09-17T21:49:28.685362+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:49:28.685362+0000","last_undegraded":"2024-09-17T21:49:28.685362+0000","last_fullsized":"2024-09-17T21:49:28.685362+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":490,"log_dups_size":0,"ondisk_log_size":490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:51:53.608463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057057109999999996,"stat_sum":{"num_bytes":7285376,"num_objects":41,"num_object_clones":4,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":22,"num_read":4858,"num_read_kb":14254,"num_write":519,"num_write_kb":2537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.b","version":"206'506","reported_seq":10537,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.765193+0000","last_change":"2024-09-17T21:49:28.670938+0000","last_active":"2024-09-17T21:49:28.765193+0000","last_peered":"2024-09-17T21:49:28.765193+0000","last_clean":"2024-09-17T21:49:28.765193+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:49:28.765193+0000","last_undegraded":"2024-09-17T21:49:28.765193+0000","last_fullsized":"2024-09-17T21:49:28.765193+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:55:37.201931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011557589999999999,"stat_sum":{"num_bytes":32895564,"num_objects":59,"num_object_clones":9,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":27,"num_read":10495,"num_read_kb":57866,"num_write":487,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.c","version":"199'537","reported_seq":18938,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.726038+0000","last_change":"2024-09-17T21:49:25.726385+0000","last_active":"2024-09-17T21:49:25.726038+0000","last_peered":"2024-09-17T21:49:25.726038+0000","last_clean":"2024-09-17T21:49:25.726038+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:49:25.726038+0000","last_undegraded":"2024-09-17T21:49:25.726038+0000","last_fullsized":"2024-09-17T21:49:25.726038+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:43:16.981796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0040170470000000002,"stat_sum":{"num_bytes":39486913,"num_objects":62,"num_object_clones":12,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":31,"num_read":18022,"num_read_kb":76883,"num_write":551,"num_write_kb":16858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.d","version":"206'775","reported_seq":9670,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.688090+0000","last_change":"2024-09-17T21:49:28.688090+0000","last_active":"2024-09-17T21:49:28.688090+0000","last_peered":"2024-09-17T21:49:28.688090+0000","last_clean":"2024-09-17T21:49:28.688090+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:49:28.688090+0000","last_undegraded":"2024-09-17T21:49:28.688090+0000","last_fullsized":"2024-09-17T21:49:28.688090+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":775,"log_dups_size":0,"ondisk_log_size":775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:54.341771+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0079892009999999996,"stat_sum":{"num_bytes":115901535,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":20,"num_read":11342,"num_read_kb":74018,"num_write":879,"num_write_kb":32170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.e","version":"195'398","reported_seq":5807,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.727213+0000","last_change":"2024-09-17T21:49:25.727473+0000","last_active":"2024-09-17T21:49:25.727213+0000","last_peered":"2024-09-17T21:49:25.727213+0000","last_clean":"2024-09-17T21:49:25.727213+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:49:25.727213+0000","last_undegraded":"2024-09-17T21:49:25.727213+0000","last_fullsized":"2024-09-17T21:49:25.727213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:49:14.569434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049814560000000004,"stat_sum":{"num_bytes":14734982,"num_objects":51,"num_object_clones":8,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":31,"num_read":5587,"num_read_kb":34401,"num_write":572,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.f","version":"203'921","reported_seq":17163,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.812705+0000","last_change":"2024-09-17T21:49:25.723902+0000","last_active":"2024-09-17T21:49:25.812705+0000","last_peered":"2024-09-17T21:49:25.812705+0000","last_clean":"2024-09-17T21:49:25.812705+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:49:25.812705+0000","last_undegraded":"2024-09-17T21:49:25.812705+0000","last_fullsized":"2024-09-17T21:49:25.812705+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":921,"log_dups_size":0,"ondisk_log_size":921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:17:22.457056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016790150000000001,"stat_sum":{"num_bytes":37204697,"num_objects":62,"num_object_clones":9,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":27,"num_read":19012,"num_read_kb":65969,"num_write":1276,"num_write_kb":30218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.10","version":"196'536","reported_seq":6871,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722508+0000","last_change":"2024-09-17T21:49:25.722965+0000","last_active":"2024-09-17T21:49:25.722508+0000","last_peered":"2024-09-17T21:49:25.722508+0000","last_clean":"2024-09-17T21:49:25.722508+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:49:25.722508+0000","last_undegraded":"2024-09-17T21:49:25.722508+0000","last_fullsized":"2024-09-17T21:49:25.722508+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":536,"log_dups_size":0,"ondisk_log_size":536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:03:05.202832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00091281399999999996,"stat_sum":{"num_bytes":35577151,"num_objects":54,"num_object_clones":4,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":24,"num_read":8170,"num_read_kb":24168,"num_write":572,"num_write_kb":5419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.11","version":"206'499","reported_seq":6110,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.684299+0000","last_change":"2024-09-17T21:49:28.684868+0000","last_active":"2024-09-17T21:49:28.684299+0000","last_peered":"2024-09-17T21:49:28.684299+0000","last_clean":"2024-09-17T21:49:28.684299+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:49:28.684299+0000","last_undegraded":"2024-09-17T21:49:28.684299+0000","last_fullsized":"2024-09-17T21:49:28.684299+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:45:54.549048+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0052626970000000002,"stat_sum":{"num_bytes":88639024,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":34,"num_read":4764,"num_read_kb":61218,"num_write":566,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.12","version":"203'360","reported_seq":12418,"reported_epoch":203,"state":"active+clean","last_fresh":"2024-09-17T21:49:25.722010+0000","last_change":"2024-09-17T21:49:25.722360+0000","last_active":"2024-09-17T21:49:25.722010+0000","last_peered":"2024-09-17T21:49:25.722010+0000","last_clean":"2024-09-17T21:49:25.722010+0000","last_became_active":"2024-09-17T21:49:22.711125+0000","last_became_peered":"2024-09-17T21:49:22.711125+0000","last_unstale":"2024-09-17T21:49:25.722010+0000","last_undegraded":"2024-09-17T21:49:25.722010+0000","last_fullsized":"2024-09-17T21:49:25.722010+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:46:00.273705+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0037584490000000001,"stat_sum":{"num_bytes":27157580,"num_objects":42,"num_object_clones":3,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":10136,"num_read_kb":51965,"num_write":441,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":54272456,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2a","length":"c"}]},{"pgid":"3.13","version":"206'443","reported_seq":10126,"reported_epoch":206,"state":"active+clean","last_fresh":"2024-09-17T21:49:28.684137+0000","last_change":"2024-09-17T21:49:28.684455+0000","last_active":"2024-09-17T21:49:28.684137+0000","last_peered":"2024-09-17T21:49:28.684137+0000","last_clean":"2024-09-17T21:49:28.684137+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:49:28.684137+0000","last_undegraded":"2024-09-17T21:49:28.684137+0000","last_fullsized":"2024-09-17T21:49:28.684137+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:49:37.178775+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.0048708349999999996,"stat_sum":{"num_bytes":38477562,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":8718,"num_read_kb":42428,"num_write":549,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1565486200,"num_objects":1840,"num_object_clones":295,"num_object_copies":3680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1840,"num_whiteouts":894,"num_read":371479,"num_read_kb":1867832,"num_write":20919,"num_write_kb":601854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10848,"num_bytes_recovered":1229762475,"num_keys_recovered":2095,"num_objects_omap":42,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675573760,"data_stored":984212955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":18623,"ondisk_log_size":18623,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607600,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":197240,"kb_used_data":143832,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94174600,"statfs":{"total":96636764160,"available":96434790400,"internally_reserved":0,"allocated":147283968,"data_stored":228466396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159821,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":171652,"kb_used_data":133988,"kb_used_omap":28,"kb_used_meta":37283,"kb_avail":94200188,"statfs":{"total":96636764160,"available":96460992512,"internally_reserved":0,"allocated":137203712,"data_stored":200431941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":38177872},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607599,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147716,"kb_used_data":98472,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94224124,"statfs":{"total":96636764160,"available":96485502976,"internally_reserved":0,"allocated":100835328,"data_stored":149272677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607599,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":141812,"kb_used_data":88416,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94230028,"statfs":{"total":96636764160,"available":96491548672,"internally_reserved":0,"allocated":90537984,"data_stored":108102663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607599,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219912,"kb_used_data":144308,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94151928,"statfs":{"total":96636764160,"available":96411574272,"internally_reserved":0,"allocated":147771392,"data_stored":213430538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640304,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":125868,"kb_used_data":72472,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94245972,"statfs":{"total":96636764160,"available":96507875328,"internally_reserved":0,"allocated":74211328,"data_stored":98265091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87502848,"data_stored":106482534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67534848,"data_stored":93020495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98390016,"data_stored":148239844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":131989504,"data_stored":196649604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":145317888,"data_stored":212383862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":144838656,"data_stored":227436616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:49:30.499 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:49:30.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:30.499 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:49:30.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:49:30.501 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:30.501 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:30.501 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.510 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:30.554 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:30.551+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:30.654 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:30.652+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:30.659 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:30.659 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:30.660 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:30.747 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:49:30.754 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:30.752+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 208, 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:49:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:49:30.741089+0000", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:48:29.752823+0000", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:49:18.682438+0000", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5804032, 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nodeep-scrub", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-17T21:49:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "luminous", 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:49:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:49:30.813 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:49:30.814 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:49:30.815 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:49:30.816 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:30.817 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:30.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:30.819 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.25, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.25, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-17T21:49:30.820 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:30.821 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:49:30.822 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "208", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:30.823 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 57, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 208, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:30.824 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:30.825 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:30.826 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.5, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.5, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.827 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 200, 2024-09-17T21:49:30.828 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:49:30.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:49:30.831 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:49:30.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 198, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:30.834 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:49:30.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 200, 2024-09-17T21:49:30.840 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:30.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.845 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 12, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 95, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 148, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 207, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 97, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.847 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:30.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/51640907", 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/51640907", 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/51640907", 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/51640907", 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.851 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 187, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.853 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:30.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.857 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 200, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:30.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:30.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.861 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.864 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.865 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 97 2024-09-17T21:49:30.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:49:30.867 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [ 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.6", 2024-09-17T21:49:30.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: 5 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [ 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pgid": "3.14", 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: 4 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.869 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 37, 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 9 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-17T21:49:30.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 37, 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:49:30.872 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.873 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.875 INFO:tasks.thrashosds.thrasher:j is {'epoch': 208, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:49:30.741089+0000', 'last_up_change': '2024-09-17T21:48:29.752823+0000', 'last_in_change': '2024-09-17T21:49:18.682438+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '208', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 57, 'snap_epoch': 208, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 200, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 198, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 200, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 12, 'last_clean_end': 95, 'up_from': 148, 'up_thru': 207, 'down_at': 97, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 51640907}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 51640907}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 51640907}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 51640907}]}, 'public_addr': '172.21.15.53:6817/51640907', 'cluster_addr': '172.21.15.53:6819/51640907', 'heartbeat_back_addr': '172.21.15.53:6823/51640907', 'heartbeat_front_addr': '172.21.15.53:6821/51640907', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 187, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 200, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 97}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.6', 'osds': [0, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.14', 'osds': [4]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 37, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:49:30.875 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.6 2024-09-17T21:49:30.875 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.6 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 175.95463563999999, 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:30.877 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:30.878 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:30.880 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.881 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:30.882 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.91818933, 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.884 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:30.886 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918168029, 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:30.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.890 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.891 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:30.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918147268, 2024-09-17T21:49:30.893 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.895 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.896 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.91812616199999, 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:30.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:30.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918105673, 2024-09-17T21:49:30.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:30.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.91808157600001, 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.907 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:30.910 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:30.911 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918059083, 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:30.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:30.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918031476, 2024-09-17T21:49:30.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:30.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:30.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 103.918006636, 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.766052861000006, 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:30.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.931 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.766029098999994, 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:30.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:30.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.765986709999993, 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.936 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:30.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:30.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710238056999998, 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.941 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:30.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710214961999995, 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:30.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:30.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710188582000001, 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710171946000003, 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:30.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710141981999996, 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.963 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710129057000003, 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:30.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:30.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 99.710118836000007, 2024-09-17T21:49:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:30.973 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:30.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:30.974 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:30.872+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:30.975 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:30.972+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:31.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:31.076 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:31.073+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:31987 3.14 3.c29b8d4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:49:30.833601+0000", 2024-09-17T21:49:31.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.25587766499999998, 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.25590070399999998, 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "waiting for peered", 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 31987 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833601+0000", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833601+0000", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833602+0000", 2024-09-17T21:49:31.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.33e-06 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833609+0000", 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6800000000000004e-06 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833625+0000", 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6365999999999999e-05 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833634+0000", 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0040000000000005e-06 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833704+0000", 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9454000000000003e-05 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:31.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:49:30.833714+0000", 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0222e-05 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-17T21:49:31.096 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:31.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:31.172 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:31.173 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:31.177 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:31.175+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:31.190 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:31.190 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:31.206 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:31.206 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:31.208 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:31.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:31.223 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:31.223 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:31.240 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:31.240 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:31.256 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:31.309 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:31.307+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.325 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:31.378 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:31.376+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.441 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.449 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:31.479 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:31.477+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.557 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.566 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:31.610 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:31.608+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.673 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.680 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:31.678+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:31.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:clear 3.6 pg_upmap mapping 2024-09-17T21:49:31.811 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:31.809+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.844 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:31.883 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:31.880+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:31.979 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:31.983 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:31.981+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:31.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:32.084 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:32.082+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.095 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.104 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:32.185 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:32.183+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:32.224 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.224 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:32.285 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:32.283+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:32.341 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.341 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.350 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:32.391 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:32.389+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:32.487 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:32.485+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:32.498 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.498 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:32.590 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:32.588+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:32.654 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.654 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.663 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:32.703 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:32.701+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:32.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:49:32.790 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:32.788+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:32.812 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.813 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.822 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:32.890 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:32.888+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:32.931 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:32.931 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:32.940 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:32.991 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:32.989+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:33.050 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:33.050 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:33.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:33.091 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:33.089+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:33.192 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:33.190+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:33.206 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:33.206 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:33.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:33.269 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:33.312 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:33.310+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:33.323 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:33.323 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:33.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:33.393 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:33.394 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:33.392+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:33.410 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:33.410 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:33.426 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:33.426 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:33.443 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:33.443 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:33.453 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:33.453 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:33.460 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:33.460 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:33.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:33.477 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:33.477 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:33.531 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:33.529+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:33.571 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:33.571 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:33.571 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:33.571 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:33.571 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:33.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:33.624 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:33.621+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:33.673 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:33.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 187.952219791, 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:33.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 187.951175331, 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:33.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:33.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 126.74999, 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:33.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:33.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.534072331999994, 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:33.715 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:33.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:33.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.331635590000005, 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:33.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:33.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:33.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:33.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:33.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:33.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.331578410000006, 2024-09-17T21:49:33.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:33.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:33.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.331007829000001, 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.739 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:33.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.330979894999999, 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.745 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:33.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:33.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:33.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:33.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:33.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.330845542999995, 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:33.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:33.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:33.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:33.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.330824007999993, 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:33.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:33.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.330663043000001, 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.764 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:33.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:33.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.330607278000002, 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:33.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:33.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.329260671, 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.776 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:33.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:33.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:33.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.329209114999998, 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.782 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.328520854000004, 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.788 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:33.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:33.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:33.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.328503803000004, 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.796 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.327565562999993, 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:33.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:33.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.327535222999998, 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.807 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:33.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 90.033137775, 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:33.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:33.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:33.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:33.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 89.132020075, 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:33.819 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:33.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:33.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:33.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:33.825 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:33.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:33.827 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:33.824+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:33.927 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:33.925+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:33.976 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:33.976 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:33.976 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:33.976 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:33.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:34.028 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:34.026+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:34.092 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:34.092 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:34.092 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:34.092 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:34.092 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:34.101 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:34.144 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:34.142+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:34.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23679101799999, 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23676675799999, 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:34.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:34.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:34.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23674087500001, 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:34.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:34.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.236715874, 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:34.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23667607499999, 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:34.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:34.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23666033000001, 2024-09-17T21:49:34.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:34.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:34.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.240 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23664879500001, 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.241 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:34.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23663849499999, 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:34.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23466725200001, 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.251 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:34.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23465410399999, 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:34.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:34.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.234643498, 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:34.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23463315999999, 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:34.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:34.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.234623153, 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:34.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23461320199999, 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:34.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23459789399999, 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:34.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:34.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23261105099999, 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:34.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:34.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.23259856199999, 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:34.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 197.232588099, 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:34.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.533252300999997, 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:34.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.178522325000003, 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:34.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.310 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:34.312 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:34.312 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:34.309+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:34.412 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:34.410+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:34.460 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:34.460 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:34.460 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:34.460 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:34.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:34.513 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:34.511+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:34.576 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:34.576 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:34.577 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:34.577 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:34.586 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:34.614 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:34.612+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:34.715 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:34.713+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764163058000001, 2024-09-17T21:49:34.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:34.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:34.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764138920999997, 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.744 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764106666000004, 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:34.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764076613, 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:34.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:34.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:34.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764053267000001, 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:34.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:34.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.764000320000001, 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.762 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:34.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.763966973999999, 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:34.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760635741000002, 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:34.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760607841000002, 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:34.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:34.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:34.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760585134999999, 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:34.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:34.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:34.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760568867000003, 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:34.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.789 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760558895999999, 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:34.793 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760546746999999, 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:34.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:34.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760537051999997, 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:34.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:34.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760527459000002, 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760517968999999, 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:34.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:34.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760508123000001, 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:34.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:34.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.760498783000003, 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:34.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:34.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:34.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 46.062285369000001, 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:34.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:34.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:34.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 46.061695397000001, 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:34.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:34.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:34.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:34.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:34.832 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:34.833 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:34.831+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:34.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:34.934 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:34.931+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:32085 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e212)", 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:49:34.704677+0000", 2024-09-17T21:49:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 0.248792924, 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.248811491, 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "waiting for peered", 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 32085 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704677+0000", 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704677+0000", 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704680+0000", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8739999999999999e-06 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704687+0000", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6699999999999994e-06 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704712+0000", 2024-09-17T21:49:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4502999999999999e-05 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704720+0000", 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1419999999999993e-06 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704791+0000", 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0675999999999999e-05 2024-09-17T21:49:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:49:34.704804+0000", 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3111e-05 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 1 2024-09-17T21:49:34.960 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:34.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:35.048 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:35.046+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:35.114 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:35.114 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:35.114 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:35.114 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:35.114 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:35.123 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:35.136 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:35.134+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:35.237 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:35.235+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:35.272 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 193.586207395, 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.273 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:35.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:35.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 193.55393242, 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.279 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:35.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 193.55369365199999, 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.285 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:35.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:35.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 144.386187522, 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.291 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:35.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 144.379285751, 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:35.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 144.37916966, 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:35.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:35.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:35.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:35.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.18216614000001, 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:35.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:35.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.313 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:35.314 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.182127345, 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:35.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:35.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:35.318 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:35.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.182113061, 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.320 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:35.321 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:35.322 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:35.323 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.181868095, 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.325 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:35.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:35.328 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.181785647, 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:35.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.331 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:35.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:35.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 110.181753837, 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:35.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:35.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.79509072099999, 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.341 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:35.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.785161741, 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:35.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.592827026999998, 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:35.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:35.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.358 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.592569605999998, 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:35.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:35.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.592536760000002, 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.592519815000003, 2024-09-17T21:49:35.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:35.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:35.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:35.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:35.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.592478318000005, 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.376 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:35.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 41.598241815000002, 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:35.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:35.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:35.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:35.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:35.387 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:35.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:35.389 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:35.386+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:35.489 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:35.487+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:35.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:35.538 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:35.538 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:35.538 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:35.547 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:35.592 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:35.590+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:35.615 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:35.615 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:35.632 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:35.632 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:35.648 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:35.649 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:35.656 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:35.656 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:35.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:35.666 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:35.666 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:35.687 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:35.687 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:35.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:49:35.690 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:35.688+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:35.707 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:35.708 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:35.790 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:35.790 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:35.791 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:35.788+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:35.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:35.891 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:35.889+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:35.910 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:35.910 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:35.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:36.003 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:36.001+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:36.066 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.067 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:36.116 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:36.114+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:36.193 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:36.191+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:36.223 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.223 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:36.294 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:36.292+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:36.352 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.352 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.361 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:36.395 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:36.393+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:36.496 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:36.494+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:36.511 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.511 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:36.598 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:36.595+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:36.628 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.628 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:36.680 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:49:36.699 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:36.696+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:36.771 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:36.771 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:36.771 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-17T21:49:36.771 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:49:36.771 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:49:36.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-17T21:49:36.792 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.792 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.799 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:36.796+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:36.801 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:36.882 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:36.882 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "" 2024-09-17T21:49:36.882 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:36.899 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:36.897+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:36.948 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:36.948 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:36.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:37.000 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:36.998+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:37.101 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:37.098+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:37.105 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.105 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:37.203 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:37.200+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:37.261 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.261 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:37.313 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:37.311+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:37.377 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.377 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.386 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:37.404 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:37.401+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:37.504 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:37.502+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:37.534 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.534 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.542 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:37.606 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:37.603+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:37.662 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.663 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:37.713 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:37.711+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:37.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:37.807 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:37.804+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:37.830 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:37.830 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:37.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:37.886 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:37.887 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:37.903 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:37.904 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:37.908 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:37.905+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:37.921 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:37.922 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:37.939 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:37.939 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:37.952 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 196.26648171599999, 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.953 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:37.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:37.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:37.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 196.23420674100001, 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:37.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:37.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 196.23396797300001, 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:37.964 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:37.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:37.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 147.06646184300001, 2024-09-17T21:49:37.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:37.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:37.975 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 147.05956007200001, 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.976 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:37.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:37.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:37.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 147.05944398099999, 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:37.981 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:37.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:37.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:37.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:37.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:37.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:38.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:38.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:38.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.064 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.86244046100001, 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.862401666, 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.069 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:38.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.86238738199999, 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.074 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:38.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.862142416, 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:38.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:38.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.862059968, 2024-09-17T21:49:38.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.862028158, 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:38.088 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:38.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:38.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.47536504200001, 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:38.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:38.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.46543606199999, 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:38.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.273101347999997, 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.106 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:38.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:38.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:38.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:38.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.272843926999997, 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.117 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:38.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.272811081, 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:38.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:38.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.272794136000002, 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:38.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:38.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:38.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 90.272752639000004, 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:38.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 44.278516136, 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:38.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:38.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.147 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.147 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:38.148 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:38.149 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:38.149 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:38.150 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:38.150 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:38.145+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:38.248 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:38.246+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:38.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:38.298 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:38.298 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:38.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.306 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:38.350 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:38.348+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:38.414 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:38.414 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:38.414 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:38.414 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:38.414 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:38.450 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:38.447+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:38.551 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:38.548+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:38.571 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:38.571 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.485311337, 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:38.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.48415300400001, 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:38.577 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:38.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.484136459, 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.582 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:38.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.484121694, 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.587 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:38.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.48409438900001, 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.593 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:38.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.598 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.484057501, 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:38.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:38.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:38.603 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.484018803, 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:38.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.48398063, 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:38.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:38.612 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.48369228999999, 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.613 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:38.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:38.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.616 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.617 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.483665694, 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:38.618 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.909127888, 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:38.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:38.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.628 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.909094573000004, 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:38.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:38.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.902004508999994, 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:38.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:38.637 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.638 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:38.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 94.901970829000007, 2024-09-17T21:49:38.640 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:38.641 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.642 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:38.644 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.813501899000002, 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:38.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.648 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.649 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:38.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.651 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.813408879999997, 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:38.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.654 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.655 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:38.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:38.658 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.810778932999995, 2024-09-17T21:49:38.659 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:38.660 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.661 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.662 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 68.467236229999997, 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.811916733000004, 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 67.806007703000006, 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:38.685 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:38.687 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:38.684+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:38.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:49:38.787 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:38.785+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:38.842 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:38.842 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:38.842 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:38.842 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:38.888 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:38.886+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:38.962 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:38.962 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:38.963 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:38.963 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:38.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:38.989 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:38.986+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:39.080 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:39.080 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:39.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:39.080 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:39.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 184.20418941899999, 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:39.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:39.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167743109, 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:39.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:39.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167721808, 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.091 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:39.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.16770104699999, 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.096 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:39.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:39.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167679941, 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.101 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:39.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167659452, 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:39.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167635355, 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:39.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167612862, 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:39.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.16758525500001, 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:39.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:39.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:39.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 112.167560415, 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.01560664, 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:39.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.015582878, 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.132 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:39.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.015540489, 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.136 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:39.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.95979183599999, 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:39.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.959768741, 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:39.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.959742361, 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.959725725, 2024-09-17T21:49:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.95969576100001, 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.959682836, 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 107.959672615, 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:39.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:39.174 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:39.176 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:39.176 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:39.173+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:39.277 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:39.274+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:39.296 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:39.296 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:39.296 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:39.296 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:39.305 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:39.377 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:39.375+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:39.454 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:39.455 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:39.455 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:39.455 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:39.455 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:39.463 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:39.507 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:39.505+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:39.572 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 193.829910493, 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.573 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 193.828866033, 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:39.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 132.62768070199999, 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:39.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:39.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:39.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:39.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.411763034, 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:39.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.209326292, 2024-09-17T21:49:39.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:39.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:39.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:39.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.603 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.209269112, 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:39.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:39.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.208698531, 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:39.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.20867059699999, 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:39.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:39.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.208536245, 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.621 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:39.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:39.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.20851471, 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:39.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:39.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:39.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.208353745, 2024-09-17T21:49:39.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.20829798, 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:39.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:39.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.206951373, 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:39.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.20689981699999, 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:39.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:39.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.206211556, 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:39.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.206194505, 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:39.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.205256265, 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:39.665 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:39.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:39.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 101.20522592499999, 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:39.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:39.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:39.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.910828476999995, 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:39.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:39.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.009710776999995, 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:39.683 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:39.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:39.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:39.688 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:39.690 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:39.690 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:39.687+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:49:39.790 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:39.788+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:39.846 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:39.846 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:39.846 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:39.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:39.891 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:39.889+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:39.962 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:39.962 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:39.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:39.992 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:39.990+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:40.079 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10488682499999, 2024-09-17T21:49:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.081 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:40.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:40.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:40.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:40.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10486256499999, 2024-09-17T21:49:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:40.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:40.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:40.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:40.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10483668200001, 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.103 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:40.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:40.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.104811681, 2024-09-17T21:49:40.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:40.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10477188199999, 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10475613700001, 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:40.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:40.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10474460200001, 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:40.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:40.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.104734302, 2024-09-17T21:49:40.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:40.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.130 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10276305900001, 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:40.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.135 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10274991099999, 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.139 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.102739305, 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:40.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:40.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10272896699999, 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:40.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:40.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10271896, 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:40.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10270900899999, 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:40.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:40.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.10269370099999, 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:40.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:40.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:40.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.100706858, 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:40.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:40.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:40.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.100694369, 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:40.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:40.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:40.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 203.100683906, 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:40.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.401348108000001, 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:40.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.046618131999999, 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:40.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:40.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:40.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:40.186 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:40.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:40.187 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:40.184+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:40.188 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:40.287 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:40.285+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:40.295 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:40.295 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:40.295 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:40.296 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:40.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:40.323 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:40.340 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:40.340 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:40.356 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:40.356 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:40.373 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:40.373 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:40.388 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:40.385+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:40.390 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:40.390 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:40.406 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:40.406 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:40.411 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:40.411 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:40.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:40.489 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:40.487+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:40.568 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:40.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:40.590 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:40.588+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:40.685 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:40.685 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:40.691 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:40.689+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:40.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:40.793 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:40.790+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:40.811 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:40.811 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:40.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:40.893 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:40.891+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:40.929 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:40.929 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:40.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:41.021 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:41.019+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:41.084 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.085 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:41.135 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:41.132+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:41.195 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:41.193+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:41.241 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.241 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:41.297 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:41.294+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.357 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.366 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:41.397 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:41.395+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:41.472 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.473 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:41.498 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:41.496+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:41.589 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.589 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.597 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:41.599 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:41.596+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:41.700 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:41.698+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:41.708 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.708 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:49:41.719 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:41.801 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:41.799+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:41.845 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.845 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:41.891 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:41.892 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:41.892 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-17T21:49:41.892 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-09-17T21:49:41.935 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:41.933+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:41.962 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:41.962 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:41.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:42.002 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:42.000+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.030 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-09-17T21:49:42.078 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:42.079 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:42.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:42.114 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:42.112+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:42.195 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:42.195 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:42.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:42.205 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:42.201+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:42.205 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.205 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.205 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.206 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.208 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-09-17T21:49:42.305 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:42.302+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:42.351 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:42.352 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:42.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.382 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.383 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.385 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.386 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-09-17T21:49:42.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:42.427 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:42.425+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:42.467 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:42.467 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:42.476 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:42.506 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:42.504+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:42.553 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:42.553 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.565 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.568 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-09-17T21:49:42.570 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:42.570 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:42.583 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:42.583 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:42.586 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:42.587 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:42.592 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:42.604 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:42.621 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:42.621 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:42.635 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:42.633+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:42.638 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:42.638 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:42.700 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:42.700 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:42.700 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.705 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:49:42.708 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:42.705+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:42.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:42.712 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.713 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-09-17T21:49:42.809 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:42.806+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:no valid command found; 10 closest matches: 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:0 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:1 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:2 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:abort 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:assert 2024-09-17T21:49:42.854 INFO:teuthology.orchestra.run.smithi007.stderr:bench [] [] [] [] 2024-09-17T21:49:42.855 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs debug_inject_read_zeros 2024-09-17T21:49:42.855 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs files list 2024-09-17T21:49:42.855 INFO:teuthology.orchestra.run.smithi007.stderr:bluefs stats 2024-09-17T21:49:42.855 INFO:teuthology.orchestra.run.smithi007.stderr:bluestore allocator dump block 2024-09-17T21:49:42.855 INFO:teuthology.orchestra.run.smithi007.stderr:Error EINVAL: invalid command 2024-09-17T21:49:42.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-17T21:49:42.857 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-17T21:49:42.862 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:42.862 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:42.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887369659000001, 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:42.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887345522000004, 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.867 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:42.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:42.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:42.871 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887313266999996, 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:42.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:42.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.875 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887283214000007, 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:42.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887259868000001, 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887206921000001, 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.887173575000006, 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.888 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883842341999994, 2024-09-17T21:49:42.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883814442000002, 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:42.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883791736000006, 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:42.901 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883775467999996, 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:42.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883765496999999, 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:42.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883753347999999, 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883743652999996, 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883734059999995, 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:42.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883724569999998, 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:42.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:42.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883714724000001, 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.932 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:42.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.936 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.883705383999995, 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:42.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:42.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.185491970000001, 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:42.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.184901998000001, 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:42.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:42.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:42.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:42.951 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:42.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:42.953 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:42.950+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:43.053 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:43.051+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:43.060 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.061 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:43.061 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:43.061 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:43.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:43.155 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:43.152+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:43.217 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.217 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:43.217 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:43.217 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:43.217 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:43.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:43.269 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:43.267+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 201.648290648, 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.335 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:43.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:43.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 201.61601567299999, 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:43.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:43.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 201.61577690499999, 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:43.348 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:43.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 152.448270775, 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.351 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:43.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.355 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:43.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 152.44136900399999, 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:43.357 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 152.441252913, 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:43.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:43.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.244249393, 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:43.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.244210598, 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.373 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:43.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:43.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:43.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.24419631400001, 2024-09-17T21:49:43.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:43.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:43.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.243951348, 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.382 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.383 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.386 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.2438689, 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:43.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:43.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:43.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 118.24383709, 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:43.391 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:43.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.85717397400001, 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.396 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:43.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:43.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:43.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:43.401 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.84724499399999, 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:43.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:43.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:43.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.654910279999996, 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:43.407 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:43.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.654652858999995, 2024-09-17T21:49:43.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:43.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:43.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:43.419 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.654620012999999, 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.420 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:43.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.425 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.654603068, 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.654561571000002, 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.431 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 49.660325067999999, 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:43.437 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:43.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.442 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:43.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:43.444 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:43.441+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:43.544 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:43.542+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:43.591 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.591 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:43.591 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:43.591 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:43.600 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:43.684 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:43.682+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:43.746 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:43.743+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:43.754 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.755 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:43.755 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:43.755 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:43.755 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:43.764 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:43.847 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:43.844+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:43.913 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.82697998099999, 2024-09-17T21:49:43.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825821648, 2024-09-17T21:49:43.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.922 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:43.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.82580510299999, 2024-09-17T21:49:43.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:43.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:43.930 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825790338, 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:43.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:43.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:43.938 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825763033, 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:43.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.941 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.942 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:43.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825726145, 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.944 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:43.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:43.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.82568744700001, 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.950 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:43.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825649274, 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:43.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:43.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.960 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825360934, 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.961 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:43.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:43.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 105.825334338, 2024-09-17T21:49:43.966 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:43.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:43.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.250796532, 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.972 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:43.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:43.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:43.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.250763217, 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:43.979 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:43.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.983 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:43.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.985 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.243673153, 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:43.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:43.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 100.243639473, 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.992 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:43.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.995 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:43.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.155170542999997, 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:43.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:43.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.000 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:44.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:44.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.155077524000006, 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:44.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:44.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:44.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.013 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 74.152447577000004, 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.014 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:44.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:44.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.808904874000007, 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:44.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:44.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:44.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.153585376999999, 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.030 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:44.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:44.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 73.147676347000001, 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.037 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:44.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.043 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:44.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:44.045 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:44.042+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:44.145 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:44.143+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:44.153 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:44.153 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:44.154 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:44.154 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:44.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:44.246 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:44.244+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:44.269 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:44.270 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:44.270 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:44.278 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:44.346 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:44.344+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:44.427 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:44.427 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 189.551576466, 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.428 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:44.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:44.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.515130156, 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:44.434 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.51510885499999, 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:44.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.443 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.51508809400001, 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:44.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:44.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.515066988, 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:44.448 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:44.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:44.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.51504649899999, 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:44.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:44.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.456 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.515022402, 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:44.459 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.460 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.514999909, 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.463 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:44.465 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.514972302, 2024-09-17T21:49:44.466 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.468 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:44.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.51494746199999, 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:44.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.362993687, 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:44.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:44.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.362969925, 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.480 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.362927536, 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.485 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.30717888300001, 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.307155788, 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:44.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.30712940799999, 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:44.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:44.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.503 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.307112772, 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.504 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:44.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.307082808, 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:44.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.512 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.307069883, 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.513 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:44.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.517 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.307059662, 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:44.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:44.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:44.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:44.522 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:44.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:44.524 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:44.521+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:44.624 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:44.621+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:44.644 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:44.645 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:44.645 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:44.645 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:44.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:44.654 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:44.724 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:44.722+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:44.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:49:44.789 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:44.789 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:44.808 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:44.808 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:44.809 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:44.809 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:44.809 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:44.811 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:44.811 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:44.818 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:44.829 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:44.846 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:44.846 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:44.861 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:44.859+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:44.866 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:44.884 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:44.884 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:44.926 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:44.924+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:44.928 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:44.928 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:44.937 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:45.027 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:45.025+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:45.085 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.085 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:45.134 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:45.132+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:45.229 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:45.227+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:45.240 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.240 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.249 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:45.330 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:45.328+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:45.396 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.396 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.405 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:45.431 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:45.429+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:45.511 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.512 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:45.565 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:45.563+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:45.628 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.628 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.632 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:45.630+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:45.636 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:49:45.733 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:45.731+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:45.793 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.793 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:45.833 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:45.831+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:45.917 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:45.917 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:45.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:45.969 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:45.967+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:46.032 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.032 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.034 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:46.032+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:46.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:46.135 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:46.133+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:46.189 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.190 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:46.236 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:46.234+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:46.337 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:46.335+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:46.345 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.345 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:46.438 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:46.436+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:46.461 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.462 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:46.554 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:46.552+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:46.618 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.618 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:46.640 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:46.637+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:46.741 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:46.738+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:46.777 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.777 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:46.841 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:46.839+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:46.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:46.897 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:46.897 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:46.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:46.947 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:46.945+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:47.021 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:47.022 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:47.039 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:47.043 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:47.041+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:47.055 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:47.055 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:47.058 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:47.058 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:47.058 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:47.058 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:47.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 192.182314878, 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.059 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:47.060 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.061 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:47.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:47.063 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.145868568, 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:47.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:47.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:47.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.14584726699999, 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.070 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:47.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.14582650600001, 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:47.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:47.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.079 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.1458054, 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.080 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:47.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:47.084 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.14578491100001, 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.085 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:47.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.089 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.145760814, 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:47.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.145738321, 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:47.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:47.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.145710714, 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:47.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 120.14568587399999, 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.104 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:47.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:47.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:47.108 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.993732099, 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.993708337, 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.113 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.117 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.993665948, 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:47.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.93791729500001, 2024-09-17T21:49:47.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:47.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:47.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.9378942, 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.127 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:47.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:47.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.93786781999999, 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.132 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.136 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.937851184, 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.137 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:47.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.93782122, 2024-09-17T21:49:47.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.937808295, 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:47.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:47.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.937798074, 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:47.151 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:47.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:47.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:47.156 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:47.157 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:47.158 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:47.158 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:47.158 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:47.158 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:47.159 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:47.154+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:47.257 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:47.255+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:47.273 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:47.273 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:47.273 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:47.282 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:47.309 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:47.309 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:47.365 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:47.363+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:47.429 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:47.429 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:47.430 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:47.430 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:47.438 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:47.479 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:47.477+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:47.560 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:47.558+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:47.588 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 201.84510347599999, 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:47.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 201.84405901599999, 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:47.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:47.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 140.64287368500001, 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.42695601699999, 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:47.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22451927500001, 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:47.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:47.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:47.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22446209500001, 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:47.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:47.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:47.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.223891514, 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.626 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22386358, 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:47.631 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:47.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:47.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.223729228, 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:47.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22370769299999, 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:47.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:47.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:47.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.223546728, 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:47.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:47.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.223490963, 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:47.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:47.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.222144356, 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:47.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:47.663 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.2220928, 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:47.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22140453900001, 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:47.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.221387488, 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:47.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:47.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:47.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:47.679 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.22044924799999, 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:47.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.220418908, 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:47.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:47.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 103.92602146, 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:47.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:47.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:47.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:47.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 103.02490376, 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:47.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:47.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:47.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:47.702 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:47.704 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:47.704 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:47.701+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:47.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:49:47.804 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:47.802+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:47.822 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:47.822 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:47.822 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:47.831 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:47.858 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:47.858 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:47.858 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-17T21:49:47.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-17T21:49:47.905 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:47.903+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:47.940 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:47.940 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:47.940 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:47.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:48.032 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:48.030+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:48.097 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:48.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12330464600001, 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:48.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:48.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.123280386, 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:48.102 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:48.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:48.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.123254503, 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:48.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:48.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:48.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12322950199999, 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:48.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:48.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:48.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12318970300001, 2024-09-17T21:49:48.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:48.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:48.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:48.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.123173958, 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:48.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:48.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.123162423, 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:48.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12315212300001, 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:48.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:48.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:48.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12118088, 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:48.134 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:48.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:48.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.121167732, 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:48.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12115712600001, 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:48.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:48.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.121146788, 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:48.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12113678099999, 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:48.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:48.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12112683000001, 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:48.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.12111152200001, 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:48.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:48.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:48.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.11912467900001, 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:48.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:48.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.11911219000001, 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 211.11910172699999, 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:48.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.419765929, 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:48.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:48.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.065035952999999, 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:48.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:48.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.191 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:48.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:48.193 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:48.190+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 225, 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "195a3125-addb-4952-bbe0-175c3f942cb7", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-09-17T21:45:33.855733+0000", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-09-17T21:49:47.716468+0000", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "last_up_change": "2024-09-17T21:48:29.752823+0000", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "last_in_change": "2024-09-17T21:49:18.682438+0000", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_num": 5799936, 2024-09-17T21:49:48.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_set": [ 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pglog_hardlimit", 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "purged_snapdirs", 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "recovery_deletes", 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "sortbitwise" 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_version": 6, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "full_ratio": 0.94999998807907104, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_snapshot": "", 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_max": 3, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "max_osd": 6, 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "require_min_compat_client": "luminous", 2024-09-17T21:49:48.195 INFO:teuthology.orchestra.run.smithi007.stdout: "min_compat_client": "jewel", 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "require_osd_release": "squid", 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "allow_crimson": false, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pools": [ 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 1, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": ".mgr", 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:46.146319+0000", 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 1, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool", 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:48.196 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 1, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 1, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 1, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 1, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 1, 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:48.197 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "15", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 0, 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:48.198 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:48.199 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:48.200 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "options": { 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_max": 32, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_min": 1 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": {} 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 6.059999942779541, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 6.059999942779541, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 0.33000001311302185, 2024-09-17T21:49:48.201 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 2, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "rbd", 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:45:50.763887+0000", 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:49:48.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 8, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 8, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 8, 2024-09-17T21:49:48.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 8, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 8, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "19", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:48.204 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 2, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 19, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:48.205 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:48.206 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 2.25, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 2.25, 2024-09-17T21:49:48.207 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 2.25, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 2.25, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_name": "pool_client.0", 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "create_time": "2024-09-17T21:46:00.113604+0000", 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags": 8193, 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-17T21:49:48.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": 1, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 2, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "min_size": 1, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_rule": 0, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_count": 0, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_target": 0, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_barrier": 0, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "is_stretch_pool": false, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "object_hash": 2, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_autoscale_mode": "off", 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num": 32, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num": 32, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_placement_num_target": 32, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_target": 32, 2024-09-17T21:49:48.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_num_pending": 32, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_pg_merge_meta": { 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "source_pgid": "0.0", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "ready_epoch": 0, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_started": 0, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_epoch_clean": 0, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "source_version": "0'0", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "target_version": "0'0" 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_change": "225", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend": "0", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "last_force_op_resend_preluminous": "0", 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "auid": 0, 2024-09-17T21:49:48.210 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_mode": "selfmanaged", 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_seq": 57, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_epoch": 225, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "pool_snaps": [], 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps": "[]", 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_bytes": 0, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "quota_max_objects": 0, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tiers": [], 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tier_of": -1, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "read_tier": -1, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "write_tier": -1, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_mode": "none", 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_bytes": 0, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "target_max_objects": 0, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-17T21:49:48.211 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_flush_age": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "cache_min_evict_age": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profile": "", 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_params": { 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "none" 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_period": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_count": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "use_gmt_hitset": true, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min_read_recency_for_promote": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min_write_recency_for_promote": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_grade_decay_rate": 0, 2024-09-17T21:49:48.212 INFO:teuthology.orchestra.run.smithi007.stdout: "hit_set_search_last_n": 0, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "grade_table": [], 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "stripe_width": 0, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "expected_num_objects": 0, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "fast_read": false, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "options": {}, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "application_metadata": { 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "rbd": {} 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "read_balance": { 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "score_type": "Fair distribution", 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "score_acting": 1.5, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "score_stable": 1.5, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "optimal_score": 1, 2024-09-17T21:49:48.213 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_acting": 1.5, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "raw_score_stable": 1.5, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity_weighted": 1, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity": 1, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "average_primary_affinity_weighted": 1 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osds": [ 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "2e180d1f-2ad9-4830-8725-c22ba929de39", 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.214 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 212, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6816", 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6817", 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6818", 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6819", 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6822", 2024-09-17T21:49:48.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6823", 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6820", 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6821", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3113291974 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6817/3113291974", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6819/3113291974", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6823/3113291974", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6821/3113291974", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.219 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "4d534322-006a-4257-b243-601aa8088f95", 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:48.220 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 11, 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 198, 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6800", 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6801", 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6802", 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6803", 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6806", 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6807", 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6804", 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6805", 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2863076940 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6801/2863076940", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6803/2863076940", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6807/2863076940", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6805/2863076940", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298", 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.226 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 200, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6808", 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6809", 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6810", 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6811", 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6814", 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6815", 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.230 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6812", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6813", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 3631167928 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6809/3631167928", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.7:6811/3631167928", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.7:6815/3631167928", 2024-09-17T21:49:48.231 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.7:6813/3631167928", 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "f9403889-7fb8-4c49-8193-55ec6ae285cc", 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 12, 2024-09-17T21:49:48.232 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 95, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 148, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 210, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 97, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6816", 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6817", 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6818", 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6819", 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6822", 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6823", 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6820", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6821", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 51640907 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6817/51640907", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6819/51640907", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6823/51640907", 2024-09-17T21:49:48.237 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6821/51640907", 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "cf4edbdc-6ec7-496f-94b6-7e6128956d63", 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:48.238 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 209, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6800", 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6801", 2024-09-17T21:49:48.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6802", 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6803", 2024-09-17T21:49:48.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6806", 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6807", 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6804", 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6805", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2074654185 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6801/2074654185", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6803/2074654185", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6807/2074654185", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6805/2074654185", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "uuid": "ebca2188-456e-4d5c-956d-6382dd41b9bd", 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up": 1, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "in": 1, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 1, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_affinity": 1, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_begin": 0, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "last_clean_end": 0, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up_from": 12, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "up_thru": 200, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "down_at": 0, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "lost_at": 0, 2024-09-17T21:49:48.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6808", 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6809", 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.245 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addrs": { 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6810", 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6811", 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addrs": { 2024-09-17T21:49:48.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6814", 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6815", 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addrs": { 2024-09-17T21:49:48.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v2", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6812", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.53:6813", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 2633163970 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.53:6809/2633163970", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_addr": "172.21.15.53:6811/2633163970", 2024-09-17T21:49:48.248 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_back_addr": "172.21.15.53:6815/2633163970", 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "heartbeat_front_addr": "172.21.15.53:6813/2633163970", 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "state": [ 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "exists", 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "up" 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd_xinfo": [ 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 0, 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.249 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.256552+0000", 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 1, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.060071+0000", 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.250 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 2, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.852414+0000", 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 3, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.251 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.699046+0000", 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 97 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 4, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.252 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.762211+0000", 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": 5, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "down_stamp": "0.000000", 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_probability": 0, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "laggy_interval": 0, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "features": 4540701547738038271, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "old_weight": 0, 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "last_purged_snaps_scrub": "2024-09-17T21:45:42.616713+0000", 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: "dead_epoch": 0 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.253 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap": [], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_items": [], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_upmap_primaries": [], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "pg_temp": [], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "primary_temp": [], 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "blocklist": {}, 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "range_blocklist": {}, 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "erasure_code_profiles": { 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "default": { 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "crush-failure-domain": "osd", 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "k": "2", 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "m": "1", 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "plugin": "jerasure", 2024-09-17T21:49:48.254 INFO:teuthology.orchestra.run.smithi007.stdout: "technique": "reed_sol_van" 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_snaps_queue": [ 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 24, 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 9 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: "new_removed_snaps": [ 2024-09-17T21:49:48.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 3, 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "snaps": [ 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "begin": 24, 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "length": 1 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "new_purged_snaps": [], 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_node_flags": {}, 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "device_class_flags": {}, 2024-09-17T21:49:48.256 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": { 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_enabled": false, 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_bucket_count": 0, 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "degraded_stretch_mode": 0, 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "recovering_stretch_mode": 0, 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode_bucket": 0 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:48.257 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:48.258 INFO:tasks.thrashosds.thrasher:j is {'epoch': 225, 'fsid': '195a3125-addb-4952-bbe0-175c3f942cb7', 'created': '2024-09-17T21:45:33.855733+0000', 'modified': '2024-09-17T21:49:47.716468+0000', 'last_up_change': '2024-09-17T21:48:29.752823+0000', 'last_in_change': '2024-09-17T21:49:18.682438+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 6, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 6, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T21:45:46.146319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-17T21:45:50.763887+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-17T21:46:00.113604+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '225', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 57, 'snap_epoch': 225, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '2e180d1f-2ad9-4830-8725-c22ba929de39', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 212, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6816', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6817', 'nonce': 3113291974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6818', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6819', 'nonce': 3113291974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6822', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6823', 'nonce': 3113291974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6820', 'nonce': 3113291974}, {'type': 'v1', 'addr': '172.21.15.7:6821', 'nonce': 3113291974}]}, 'public_addr': '172.21.15.7:6817/3113291974', 'cluster_addr': '172.21.15.7:6819/3113291974', 'heartbeat_back_addr': '172.21.15.7:6823/3113291974', 'heartbeat_front_addr': '172.21.15.7:6821/3113291974', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '4d534322-006a-4257-b243-601aa8088f95', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 11, 'up_thru': 198, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6800', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6801', 'nonce': 2863076940}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6802', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6803', 'nonce': 2863076940}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6806', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6807', 'nonce': 2863076940}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6804', 'nonce': 2863076940}, {'type': 'v1', 'addr': '172.21.15.7:6805', 'nonce': 2863076940}]}, 'public_addr': '172.21.15.7:6801/2863076940', 'cluster_addr': '172.21.15.7:6803/2863076940', 'heartbeat_back_addr': '172.21.15.7:6807/2863076940', 'heartbeat_front_addr': '172.21.15.7:6805/2863076940', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 200, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6808', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6809', 'nonce': 3631167928}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6810', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6811', 'nonce': 3631167928}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6814', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6815', 'nonce': 3631167928}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6812', 'nonce': 3631167928}, {'type': 'v1', 'addr': '172.21.15.7:6813', 'nonce': 3631167928}]}, 'public_addr': '172.21.15.7:6809/3631167928', 'cluster_addr': '172.21.15.7:6811/3631167928', 'heartbeat_back_addr': '172.21.15.7:6815/3631167928', 'heartbeat_front_addr': '172.21.15.7:6813/3631167928', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'f9403889-7fb8-4c49-8193-55ec6ae285cc', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 12, 'last_clean_end': 95, 'up_from': 148, 'up_thru': 210, 'down_at': 97, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 51640907}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 51640907}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 51640907}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 51640907}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 51640907}]}, 'public_addr': '172.21.15.53:6817/51640907', 'cluster_addr': '172.21.15.53:6819/51640907', 'heartbeat_back_addr': '172.21.15.53:6823/51640907', 'heartbeat_front_addr': '172.21.15.53:6821/51640907', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'cf4edbdc-6ec7-496f-94b6-7e6128956d63', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 209, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 2074654185}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 2074654185}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 2074654185}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 2074654185}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 2074654185}]}, 'public_addr': '172.21.15.53:6801/2074654185', 'cluster_addr': '172.21.15.53:6803/2074654185', 'heartbeat_back_addr': '172.21.15.53:6807/2074654185', 'heartbeat_front_addr': '172.21.15.53:6805/2074654185', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'ebca2188-456e-4d5c-956d-6382dd41b9bd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 12, 'up_thru': 200, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 2633163970}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 2633163970}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 2633163970}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 2633163970}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 2633163970}]}, 'public_addr': '172.21.15.53:6809/2633163970', 'cluster_addr': '172.21.15.53:6811/2633163970', 'heartbeat_back_addr': '172.21.15.53:6815/2633163970', 'heartbeat_front_addr': '172.21.15.53:6813/2633163970', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.256552+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.060071+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.852414+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.699046+0000', 'dead_epoch': 97}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.762211+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-17T21:45:42.616713+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 9}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-17T21:49:48.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:49:48.301 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:48.299+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:48.366 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:48.366 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:48.366 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:48.375 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:48.416 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:48.414+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:48.494 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:48.491+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:48.522 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:48.523 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:48.523 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:48.531 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:48.551 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:49:48.551 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:49:48.562 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-17T21:49:48.140700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1167936918,"num_objects":1429,"num_object_clones":178,"num_object_copies":2858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1429,"num_whiteouts":820,"num_read":375416,"num_read_kb":1871201,"num_write":21835,"num_write_kb":604477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11120,"num_bytes_recovered":1259320791,"num_keys_recovered":2095,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19788,"ondisk_log_size":19788,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":848280,"kb_used_data":525896,"kb_used_omap":36,"kb_used_meta":322203,"kb_avail":565382760,"statfs":{"total":579820584960,"available":578951946240,"internally_reserved":0,"allocated":538517504,"data_stored":813611909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":329936169},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-47630745,"num_objects":-82,"num_object_clones":-25,"num_object_copies":-164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-82,"num_whiteouts":-17,"num_read":1087,"num_read_kb":928,"num_write":193,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000722"},"pg_stats":[{"pgid":"3.1f","version":"221'375","reported_seq":11612,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.724972+0000","last_change":"2024-09-17T21:49:45.725844+0000","last_active":"2024-09-17T21:49:45.724972+0000","last_peered":"2024-09-17T21:49:45.724972+0000","last_clean":"2024-09-17T21:49:45.724972+0000","last_became_active":"2024-09-17T21:49:23.715014+0000","last_became_peered":"2024-09-17T21:49:23.715014+0000","last_unstale":"2024-09-17T21:49:45.724972+0000","last_undegraded":"2024-09-17T21:49:45.724972+0000","last_fullsized":"2024-09-17T21:49:45.724972+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:33:02.113374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0043884149999999997,"stat_sum":{"num_bytes":39067189,"num_objects":47,"num_object_clones":3,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":28,"num_read":11135,"num_read_kb":70768,"num_write":347,"num_write_kb":11994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.1e","version":"222'547","reported_seq":20332,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.717424+0000","last_change":"2024-09-17T21:49:45.719073+0000","last_active":"2024-09-17T21:49:45.717424+0000","last_peered":"2024-09-17T21:49:45.717424+0000","last_clean":"2024-09-17T21:49:45.717424+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:49:45.717424+0000","last_undegraded":"2024-09-17T21:49:45.717424+0000","last_fullsized":"2024-09-17T21:49:45.717424+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:33:17.171160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036234980000000002,"stat_sum":{"num_bytes":94536208,"num_objects":62,"num_object_clones":13,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":32,"num_read":18882,"num_read_kb":109170,"num_write":422,"num_write_kb":34493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.1d","version":"223'388","reported_seq":5352,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.863327+0000","last_change":"2024-09-17T21:49:45.727648+0000","last_active":"2024-09-17T21:49:45.863327+0000","last_peered":"2024-09-17T21:49:45.863327+0000","last_clean":"2024-09-17T21:49:45.863327+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:49:45.863327+0000","last_undegraded":"2024-09-17T21:49:45.863327+0000","last_fullsized":"2024-09-17T21:49:45.863327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":388,"log_dups_size":0,"ondisk_log_size":388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:14:20.540363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003847343,"stat_sum":{"num_bytes":16869449,"num_objects":43,"num_object_clones":4,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":28,"num_read":4901,"num_read_kb":51124,"num_write":644,"num_write_kb":31819,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.1c","version":"223'597","reported_seq":18309,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.808028+0000","last_change":"2024-09-17T21:49:45.734550+0000","last_active":"2024-09-17T21:49:45.808028+0000","last_peered":"2024-09-17T21:49:45.808028+0000","last_clean":"2024-09-17T21:49:45.808028+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:49:45.808028+0000","last_undegraded":"2024-09-17T21:49:45.808028+0000","last_fullsized":"2024-09-17T21:49:45.808028+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:02:34.057821+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.010705381999999999,"stat_sum":{"num_bytes":62688821,"num_objects":45,"num_object_clones":7,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":22,"num_read":19717,"num_read_kb":86893,"num_write":647,"num_write_kb":39749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.1b","version":"223'447","reported_seq":16839,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.728727+0000","last_change":"2024-09-17T21:49:45.728727+0000","last_active":"2024-09-17T21:49:45.728727+0000","last_peered":"2024-09-17T21:49:45.728727+0000","last_clean":"2024-09-17T21:49:45.728727+0000","last_became_active":"2024-09-17T21:48:08.675352+0000","last_became_peered":"2024-09-17T21:48:08.675352+0000","last_unstale":"2024-09-17T21:49:45.728727+0000","last_undegraded":"2024-09-17T21:49:45.728727+0000","last_fullsized":"2024-09-17T21:49:45.728727+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:59:33.338572+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0070226739999999996,"stat_sum":{"num_bytes":47815680,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":20,"num_read":14408,"num_read_kb":43491,"num_write":360,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.1a","version":"221'665","reported_seq":11630,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:49:44.842628+0000","last_change":"2024-09-17T21:49:44.717845+0000","last_active":"2024-09-17T21:49:44.842628+0000","last_peered":"2024-09-17T21:49:44.842628+0000","last_clean":"2024-09-17T21:49:44.842628+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:49:44.842628+0000","last_undegraded":"2024-09-17T21:49:44.842628+0000","last_fullsized":"2024-09-17T21:49:44.842628+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":665,"log_dups_size":0,"ondisk_log_size":665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:52:26.690695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069565499999999999,"stat_sum":{"num_bytes":60685946,"num_objects":45,"num_object_clones":5,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":25,"num_read":13605,"num_read_kb":82172,"num_write":693,"num_write_kb":25550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"3.19","version":"217'444","reported_seq":5029,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.683318+0000","last_change":"2024-09-17T21:49:43.683749+0000","last_active":"2024-09-17T21:49:43.683318+0000","last_peered":"2024-09-17T21:49:43.683318+0000","last_clean":"2024-09-17T21:49:43.683318+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:49:43.683318+0000","last_undegraded":"2024-09-17T21:49:43.683318+0000","last_fullsized":"2024-09-17T21:49:43.683318+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":444,"log_dups_size":0,"ondisk_log_size":444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:26:52.158221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001463911,"stat_sum":{"num_bytes":26772016,"num_objects":48,"num_object_clones":6,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":29,"num_read":4656,"num_read_kb":34114,"num_write":448,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.18","version":"223'662","reported_seq":13107,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.813036+0000","last_change":"2024-09-17T21:49:45.717448+0000","last_active":"2024-09-17T21:49:45.813036+0000","last_peered":"2024-09-17T21:49:45.813036+0000","last_clean":"2024-09-17T21:49:45.813036+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:49:45.813036+0000","last_undegraded":"2024-09-17T21:49:45.813036+0000","last_fullsized":"2024-09-17T21:49:45.813036+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:19.185525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020542949999999998,"stat_sum":{"num_bytes":41489026,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":27,"num_read":12285,"num_read_kb":68956,"num_write":769,"num_write_kb":13800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.17","version":"223'374","reported_seq":7680,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.724888+0000","last_change":"2024-09-17T21:49:45.724888+0000","last_active":"2024-09-17T21:49:45.724888+0000","last_peered":"2024-09-17T21:49:45.724888+0000","last_clean":"2024-09-17T21:49:45.724888+0000","last_became_active":"2024-09-17T21:48:10.691486+0000","last_became_peered":"2024-09-17T21:48:10.691486+0000","last_unstale":"2024-09-17T21:49:45.724888+0000","last_undegraded":"2024-09-17T21:49:45.724888+0000","last_fullsized":"2024-09-17T21:49:45.724888+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:47:33.316015+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003681948,"stat_sum":{"num_bytes":7298560,"num_objects":36,"num_object_clones":2,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":27,"num_read":7903,"num_read_kb":32043,"num_write":342,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.16","version":"220'980","reported_seq":11773,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.732123+0000","last_change":"2024-09-17T21:49:45.732689+0000","last_active":"2024-09-17T21:49:45.732123+0000","last_peered":"2024-09-17T21:49:45.732123+0000","last_clean":"2024-09-17T21:49:45.732123+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:49:45.732123+0000","last_undegraded":"2024-09-17T21:49:45.732123+0000","last_fullsized":"2024-09-17T21:49:45.732123+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":980,"log_dups_size":0,"ondisk_log_size":980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:27:39.318878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0085918490000000004,"stat_sum":{"num_bytes":19639331,"num_objects":54,"num_object_clones":6,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":37,"num_read":14153,"num_read_kb":74137,"num_write":1018,"num_write_kb":17165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.15","version":"219'690","reported_seq":9722,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.674794+0000","last_change":"2024-09-17T21:49:43.675342+0000","last_active":"2024-09-17T21:49:43.674794+0000","last_peered":"2024-09-17T21:49:43.674794+0000","last_clean":"2024-09-17T21:49:43.674794+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:49:43.674794+0000","last_undegraded":"2024-09-17T21:49:43.674794+0000","last_fullsized":"2024-09-17T21:49:43.674794+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":690,"log_dups_size":0,"ondisk_log_size":690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:10:57.130297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016982379999999999,"stat_sum":{"num_bytes":20616739,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":24,"num_read":16434,"num_read_kb":59390,"num_write":660,"num_write_kb":14183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.14","version":"215'486","reported_seq":19188,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.674149+0000","last_change":"2024-09-17T21:49:43.674736+0000","last_active":"2024-09-17T21:49:43.674149+0000","last_peered":"2024-09-17T21:49:43.674149+0000","last_clean":"2024-09-17T21:49:43.674149+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:49:43.674149+0000","last_undegraded":"2024-09-17T21:49:43.674149+0000","last_fullsized":"2024-09-17T21:49:43.674149+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:29:14.610067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001754757,"stat_sum":{"num_bytes":83551340,"num_objects":57,"num_object_clones":12,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":24,"num_read":18241,"num_read_kb":143650,"num_write":550,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":58,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.672836+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:49:43.672836+0000","last_peered":"2024-09-17T21:49:43.672836+0000","last_clean":"2024-09-17T21:49:43.672836+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:49:43.672836+0000","last_undegraded":"2024-09-17T21:49:43.672836+0000","last_fullsized":"2024-09-17T21:49:43.672836+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"223'721","reported_seq":10674,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.726680+0000","last_change":"2024-09-17T21:49:45.726680+0000","last_active":"2024-09-17T21:49:45.726680+0000","last_peered":"2024-09-17T21:49:45.726680+0000","last_clean":"2024-09-17T21:49:45.726680+0000","last_became_active":"2024-09-17T21:49:35.676534+0000","last_became_peered":"2024-09-17T21:49:35.676534+0000","last_unstale":"2024-09-17T21:49:45.726680+0000","last_undegraded":"2024-09-17T21:49:45.726680+0000","last_fullsized":"2024-09-17T21:49:45.726680+0000","mapping_epoch":212,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:37:38.936980+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0054299320000000002,"stat_sum":{"num_bytes":28756007,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":26,"num_read":12158,"num_read_kb":53859,"num_write":606,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":451,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.682182+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:49:43.682182+0000","last_peered":"2024-09-17T21:49:43.682182+0000","last_clean":"2024-09-17T21:49:43.682182+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:49:43.682182+0000","last_undegraded":"2024-09-17T21:49:43.682182+0000","last_fullsized":"2024-09-17T21:49:43.682182+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"223'545","reported_seq":6719,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.720741+0000","last_change":"2024-09-17T21:49:45.721077+0000","last_active":"2024-09-17T21:49:45.720741+0000","last_peered":"2024-09-17T21:49:45.720741+0000","last_clean":"2024-09-17T21:49:45.720741+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:49:45.720741+0000","last_undegraded":"2024-09-17T21:49:45.720741+0000","last_fullsized":"2024-09-17T21:49:45.720741+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:02:38.526475+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.005828153,"stat_sum":{"num_bytes":33344000,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":26,"num_read":6415,"num_read_kb":43107,"num_write":634,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":58,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.672632+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:49:43.672632+0000","last_peered":"2024-09-17T21:49:43.672632+0000","last_clean":"2024-09-17T21:49:43.672632+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:49:43.672632+0000","last_undegraded":"2024-09-17T21:49:43.672632+0000","last_fullsized":"2024-09-17T21:49:43.672632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"220'659","reported_seq":19232,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.682468+0000","last_change":"2024-09-17T21:49:43.683240+0000","last_active":"2024-09-17T21:49:43.682468+0000","last_peered":"2024-09-17T21:49:43.682468+0000","last_clean":"2024-09-17T21:49:43.682468+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:49:43.682468+0000","last_undegraded":"2024-09-17T21:49:43.682468+0000","last_fullsized":"2024-09-17T21:49:43.682468+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:43:34.292992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079981900000000001,"stat_sum":{"num_bytes":61194240,"num_objects":47,"num_object_clones":8,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":26,"num_read":18868,"num_read_kb":88401,"num_write":480,"num_write_kb":12137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":459,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.681772+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:49:43.681772+0000","last_peered":"2024-09-17T21:49:43.681772+0000","last_clean":"2024-09-17T21:49:43.681772+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:49:43.681772+0000","last_undegraded":"2024-09-17T21:49:43.681772+0000","last_fullsized":"2024-09-17T21:49:43.681772+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"223'508","reported_seq":12813,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.728388+0000","last_change":"2024-09-17T21:49:45.728388+0000","last_active":"2024-09-17T21:49:45.728388+0000","last_peered":"2024-09-17T21:49:45.728388+0000","last_clean":"2024-09-17T21:49:45.728388+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:49:45.728388+0000","last_undegraded":"2024-09-17T21:49:45.728388+0000","last_fullsized":"2024-09-17T21:49:45.728388+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":508,"log_dups_size":0,"ondisk_log_size":508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:41.079105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044138069999999996,"stat_sum":{"num_bytes":25948704,"num_objects":43,"num_object_clones":3,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":27,"num_read":11408,"num_read_kb":39484,"num_write":502,"num_write_kb":17810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.3","version":"222'888","reported_seq":11504,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:49:44.866804+0000","last_change":"2024-09-17T21:49:44.718525+0000","last_active":"2024-09-17T21:49:44.866804+0000","last_peered":"2024-09-17T21:49:44.866804+0000","last_clean":"2024-09-17T21:49:44.866804+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:49:44.866804+0000","last_undegraded":"2024-09-17T21:49:44.866804+0000","last_fullsized":"2024-09-17T21:49:44.866804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":888,"log_dups_size":0,"ondisk_log_size":888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:33:57.794942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088044399999999997,"stat_sum":{"num_bytes":16372148,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":17,"num_read":13953,"num_read_kb":38533,"num_write":1125,"num_write_kb":10602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"2.2","version":"19'2","reported_seq":430,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.723524+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:49:45.723524+0000","last_peered":"2024-09-17T21:49:45.723524+0000","last_clean":"2024-09-17T21:49:45.723524+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:49:45.723524+0000","last_undegraded":"2024-09-17T21:49:45.723524+0000","last_fullsized":"2024-09-17T21:49:45.723524+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"219'436","reported_seq":7173,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.682532+0000","last_change":"2024-09-17T21:49:43.683248+0000","last_active":"2024-09-17T21:49:43.682532+0000","last_peered":"2024-09-17T21:49:43.682532+0000","last_clean":"2024-09-17T21:49:43.682532+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:49:43.682532+0000","last_undegraded":"2024-09-17T21:49:43.682532+0000","last_fullsized":"2024-09-17T21:49:43.682532+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:46:19.470776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074319000000000002,"stat_sum":{"num_bytes":17318011,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":26,"num_read":10028,"num_read_kb":30205,"num_write":347,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":86,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.715245+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:49:45.715245+0000","last_peered":"2024-09-17T21:49:45.715245+0000","last_clean":"2024-09-17T21:49:45.715245+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:49:45.715245+0000","last_undegraded":"2024-09-17T21:49:45.715245+0000","last_fullsized":"2024-09-17T21:49:45.715245+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"223'1846","reported_seq":9018,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.723551+0000","last_change":"2024-09-17T21:49:45.724041+0000","last_active":"2024-09-17T21:49:45.723551+0000","last_peered":"2024-09-17T21:49:45.723551+0000","last_clean":"2024-09-17T21:49:45.723551+0000","last_became_active":"2024-09-17T21:48:10.690220+0000","last_became_peered":"2024-09-17T21:48:10.690220+0000","last_unstale":"2024-09-17T21:49:45.723551+0000","last_undegraded":"2024-09-17T21:49:45.723551+0000","last_fullsized":"2024-09-17T21:49:45.723551+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1846,"log_dups_size":0,"ondisk_log_size":1846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:51:00.602918+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0041418519999999997,"stat_sum":{"num_bytes":13060911,"num_objects":46,"num_object_clones":4,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":29,"num_read":15606,"num_read_kb":56872,"num_write":2727,"num_write_kb":12566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":58,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.672436+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:49:43.672436+0000","last_peered":"2024-09-17T21:49:43.672436+0000","last_clean":"2024-09-17T21:49:43.672436+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:49:43.672436+0000","last_undegraded":"2024-09-17T21:49:43.672436+0000","last_fullsized":"2024-09-17T21:49:43.672436+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"213'492","reported_seq":5611,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.683389+0000","last_change":"2024-09-17T21:49:43.683770+0000","last_active":"2024-09-17T21:49:43.683389+0000","last_peered":"2024-09-17T21:49:43.683389+0000","last_clean":"2024-09-17T21:49:43.683389+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:49:43.683389+0000","last_undegraded":"2024-09-17T21:49:43.683389+0000","last_fullsized":"2024-09-17T21:49:43.683389+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:03:10.566679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013282739999999999,"stat_sum":{"num_bytes":35470398,"num_objects":44,"num_object_clones":5,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":5927,"num_read_kb":32230,"num_write":549,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"2.3","version":"17'1","reported_seq":447,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.723632+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:49:45.723632+0000","last_peered":"2024-09-17T21:49:45.723632+0000","last_clean":"2024-09-17T21:49:45.723632+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:49:45.723632+0000","last_undegraded":"2024-09-17T21:49:45.723632+0000","last_fullsized":"2024-09-17T21:49:45.723632+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":572,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.672452+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:49:43.672452+0000","last_peered":"2024-09-17T21:49:43.672452+0000","last_clean":"2024-09-17T21:49:43.672452+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:49:43.672452+0000","last_undegraded":"2024-09-17T21:49:43.672452+0000","last_fullsized":"2024-09-17T21:49:43.672452+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"220'448","reported_seq":7151,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.719182+0000","last_change":"2024-09-17T21:49:45.719947+0000","last_active":"2024-09-17T21:49:45.719182+0000","last_peered":"2024-09-17T21:49:45.719182+0000","last_clean":"2024-09-17T21:49:45.719182+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:49:45.719182+0000","last_undegraded":"2024-09-17T21:49:45.719182+0000","last_fullsized":"2024-09-17T21:49:45.719182+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":448,"log_dups_size":0,"ondisk_log_size":448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:24:08.557542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0044150889999999996,"stat_sum":{"num_bytes":25878560,"num_objects":44,"num_object_clones":4,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":25,"num_read":7257,"num_read_kb":58601,"num_write":500,"num_write_kb":6067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.9","version":"221'813","reported_seq":10963,"reported_epoch":222,"state":"active+clean","last_fresh":"2024-09-17T21:49:44.720984+0000","last_change":"2024-09-17T21:49:44.721308+0000","last_active":"2024-09-17T21:49:44.720984+0000","last_peered":"2024-09-17T21:49:44.720984+0000","last_clean":"2024-09-17T21:49:44.720984+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:49:44.720984+0000","last_undegraded":"2024-09-17T21:49:44.720984+0000","last_fullsized":"2024-09-17T21:49:44.720984+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:08:32.854743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085914599999999998,"stat_sum":{"num_bytes":39485530,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":23,"num_read":15726,"num_read_kb":70295,"num_write":772,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"6"}]},{"pgid":"3.a","version":"221'514","reported_seq":5223,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.674044+0000","last_change":"2024-09-17T21:49:43.674044+0000","last_active":"2024-09-17T21:49:43.674044+0000","last_peered":"2024-09-17T21:49:43.674044+0000","last_clean":"2024-09-17T21:49:43.674044+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:49:43.674044+0000","last_undegraded":"2024-09-17T21:49:43.674044+0000","last_fullsized":"2024-09-17T21:49:43.674044+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:52.799614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001290802,"stat_sum":{"num_bytes":4583502,"num_objects":34,"num_object_clones":1,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":4895,"num_read_kb":14284,"num_write":539,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.b","version":"220'518","reported_seq":10634,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.682975+0000","last_change":"2024-09-17T21:49:43.683326+0000","last_active":"2024-09-17T21:49:43.682975+0000","last_peered":"2024-09-17T21:49:43.682975+0000","last_clean":"2024-09-17T21:49:43.682975+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:49:43.682975+0000","last_undegraded":"2024-09-17T21:49:43.682975+0000","last_fullsized":"2024-09-17T21:49:43.682975+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:40:00.503471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001143813,"stat_sum":{"num_bytes":26105932,"num_objects":49,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":26,"num_read":10495,"num_read_kb":57866,"num_write":493,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.c","version":"223'587","reported_seq":19181,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.731265+0000","last_change":"2024-09-17T21:49:45.731902+0000","last_active":"2024-09-17T21:49:45.731265+0000","last_peered":"2024-09-17T21:49:45.731265+0000","last_clean":"2024-09-17T21:49:45.731265+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:49:45.731265+0000","last_undegraded":"2024-09-17T21:49:45.731265+0000","last_fullsized":"2024-09-17T21:49:45.731265+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":587,"log_dups_size":0,"ondisk_log_size":587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:27.797710+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0078147649999999996,"stat_sum":{"num_bytes":17445482,"num_objects":35,"num_object_clones":3,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":24,"num_read":18203,"num_read_kb":77038,"num_write":578,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.d","version":"219'780","reported_seq":9762,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.673870+0000","last_change":"2024-09-17T21:49:43.674442+0000","last_active":"2024-09-17T21:49:43.673870+0000","last_peered":"2024-09-17T21:49:43.673870+0000","last_clean":"2024-09-17T21:49:43.673870+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:49:43.673870+0000","last_undegraded":"2024-09-17T21:49:43.673870+0000","last_fullsized":"2024-09-17T21:49:43.673870+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":780,"log_dups_size":0,"ondisk_log_size":780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:30.993903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001526955,"stat_sum":{"num_bytes":107205199,"num_objects":48,"num_object_clones":11,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":19,"num_read":11345,"num_read_kb":74019,"num_write":882,"num_write_kb":32170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.e","version":"223'424","reported_seq":5962,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.865935+0000","last_change":"2024-09-17T21:49:45.733205+0000","last_active":"2024-09-17T21:49:45.865935+0000","last_peered":"2024-09-17T21:49:45.865935+0000","last_clean":"2024-09-17T21:49:45.865935+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:49:45.865935+0000","last_undegraded":"2024-09-17T21:49:45.865935+0000","last_fullsized":"2024-09-17T21:49:45.865935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:13:47.320910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089026279999999992,"stat_sum":{"num_bytes":9147904,"num_objects":38,"num_object_clones":3,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":28,"num_read":5627,"num_read_kb":34429,"num_write":590,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.f","version":"223'938","reported_seq":17307,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.727517+0000","last_change":"2024-09-17T21:49:45.727845+0000","last_active":"2024-09-17T21:49:45.727517+0000","last_peered":"2024-09-17T21:49:45.727517+0000","last_clean":"2024-09-17T21:49:45.727517+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:49:45.727517+0000","last_undegraded":"2024-09-17T21:49:45.727517+0000","last_fullsized":"2024-09-17T21:49:45.727517+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":938,"log_dups_size":0,"ondisk_log_size":938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:35:59.181946+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0040929750000000004,"stat_sum":{"num_bytes":29868653,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":24,"num_read":19026,"num_read_kb":65975,"num_write":1285,"num_write_kb":30218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.10","version":"223'565","reported_seq":7118,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.829144+0000","last_change":"2024-09-17T21:49:45.727164+0000","last_active":"2024-09-17T21:49:45.829144+0000","last_peered":"2024-09-17T21:49:45.829144+0000","last_clean":"2024-09-17T21:49:45.829144+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:49:45.829144+0000","last_undegraded":"2024-09-17T21:49:45.829144+0000","last_fullsized":"2024-09-17T21:49:45.829144+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:17:33.492331+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0036053019999999999,"stat_sum":{"num_bytes":33212207,"num_objects":46,"num_object_clones":2,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":22,"num_read":8425,"num_read_kb":24384,"num_write":592,"num_write_kb":5425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.11","version":"219'517","reported_seq":6211,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.674506+0000","last_change":"2024-09-17T21:49:43.675058+0000","last_active":"2024-09-17T21:49:43.674506+0000","last_peered":"2024-09-17T21:49:43.674506+0000","last_clean":"2024-09-17T21:49:43.674506+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:49:43.674506+0000","last_undegraded":"2024-09-17T21:49:43.674506+0000","last_fullsized":"2024-09-17T21:49:43.674506+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":517,"log_dups_size":0,"ondisk_log_size":517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:35:23.231747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002400148,"stat_sum":{"num_bytes":66662960,"num_objects":57,"num_object_clones":13,"num_object_copies":114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57,"num_whiteouts":33,"num_read":4764,"num_read_kb":61218,"num_write":574,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]},{"pgid":"3.12","version":"214'368","reported_seq":12546,"reported_epoch":223,"state":"active+clean","last_fresh":"2024-09-17T21:49:45.720046+0000","last_change":"2024-09-17T21:49:45.720729+0000","last_active":"2024-09-17T21:49:45.720046+0000","last_peered":"2024-09-17T21:49:45.720046+0000","last_clean":"2024-09-17T21:49:45.720046+0000","last_became_active":"2024-09-17T21:49:22.711125+0000","last_became_peered":"2024-09-17T21:49:22.711125+0000","last_unstale":"2024-09-17T21:49:45.720046+0000","last_undegraded":"2024-09-17T21:49:45.720046+0000","last_fullsized":"2024-09-17T21:49:45.720046+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:15:38.586017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095253000000000004,"stat_sum":{"num_bytes":20325964,"num_objects":34,"num_object_clones":2,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":10136,"num_read_kb":51965,"num_write":447,"num_write_kb":24480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":542,"num_bytes_recovered":54272456,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1a","length":"7"}]},{"pgid":"3.13","version":"217'446","reported_seq":10214,"reported_epoch":221,"state":"active+clean","last_fresh":"2024-09-17T21:49:43.673080+0000","last_change":"2024-09-17T21:49:43.673789+0000","last_active":"2024-09-17T21:49:43.673080+0000","last_peered":"2024-09-17T21:49:43.673080+0000","last_clean":"2024-09-17T21:49:43.673080+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:49:43.673080+0000","last_undegraded":"2024-09-17T21:49:43.673080+0000","last_fullsized":"2024-09-17T21:49:43.673080+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:09.476952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011089610000000001,"stat_sum":{"num_bytes":34929914,"num_objects":46,"num_object_clones":6,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":27,"num_read":8718,"num_read_kb":42428,"num_write":552,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1c","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1167346531,"num_objects":1425,"num_object_clones":178,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":820,"num_read":375300,"num_read_kb":1871101,"num_write":21674,"num_write_kb":602059,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11102,"num_bytes_recovered":1254597847,"num_keys_recovered":2095,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":518082560,"data_stored":802457824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":19668,"ondisk_log_size":19668,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607604,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":148452,"kb_used_data":90892,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94223388,"statfs":{"total":96636764160,"available":96484749312,"internally_reserved":0,"allocated":93073408,"data_stored":156974462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159824,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":168704,"kb_used_data":131372,"kb_used_omap":28,"kb_used_meta":37283,"kb_avail":94203136,"statfs":{"total":96636764160,"available":96464011264,"internally_reserved":0,"allocated":134524928,"data_stored":202458174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":38177872},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607603,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134640,"kb_used_data":81236,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94237200,"statfs":{"total":96636764160,"available":96498892800,"internally_reserved":0,"allocated":83185664,"data_stored":128929122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607603,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":97332,"kb_used_data":39764,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94274508,"statfs":{"total":96636764160,"available":96537096192,"internally_reserved":0,"allocated":40718336,"data_stored":49741729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607603,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":180868,"kb_used_data":117748,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94190972,"statfs":{"total":96636764160,"available":96451555328,"internally_reserved":0,"allocated":120573952,"data_stored":182896905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640307,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118284,"kb_used_data":64884,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94253556,"statfs":{"total":96636764160,"available":96515641344,"internally_reserved":0,"allocated":66441216,"data_stored":92611517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":37437440,"data_stored":48015381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":63774720,"data_stored":91486228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":80498688,"data_stored":127793142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":129130496,"data_stored":198601643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":116858880,"data_stored":180722967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90382336,"data_stored":155838463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:49:48.564 INFO:tasks.thrashosds.thrasher:Setting 3.12 to [1, 3] 2024-09-17T21:49:48.564 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.12', '1', '3'] 2024-09-17T21:49:48.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.12 1 3 2024-09-17T21:49:48.615 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:48.613+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:48.675 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.708012590999999, 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:48.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707988454000002, 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:48.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:48.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707956198999995, 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:48.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:48.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707926146000005, 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:48.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:48.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:48.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707902799999999, 2024-09-17T21:49:48.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:48.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707849852999999, 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:48.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.707816507000004, 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.711 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:48.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704485274000007, 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.716 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:48.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:48.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:48.719 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704457374, 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.720 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:48.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704434668000005, 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:48.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:48.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:48.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704418399999994, 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:48.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:48.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704408428999997, 2024-09-17T21:49:48.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:48.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704396279999997, 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:48.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704386584999995, 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704376991999993, 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:48.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:48.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704367501999997, 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:48.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704357655999999, 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:48.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:48.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.704348315999994, 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:48.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.006134901999999, 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:48.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:48.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.005544929999999, 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:48.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:48.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:48.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:48.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:48.775 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:48.777 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:48.774+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:48.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:48.878 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:48.876+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:48.887 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:48.887 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:48.887 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:48.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:48.980 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:48.978+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:49.044 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:49.044 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:49.044 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:49.052 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:49.093 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:49.091+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:49.181 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:49.179+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:49.202 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:49.202 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 207.51620754300001, 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.203 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:49.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 207.483932568, 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:49.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:49.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:49.214 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 207.4836938, 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:49.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:49.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 158.31618767000001, 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.220 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:49.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 158.309285899, 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:49.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:49.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 158.30916980800001, 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.231 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:49.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.112166288, 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.236 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:49.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:49.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.240 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.112127493, 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:49.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.112113209, 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:49.245 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:49.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:49.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.111868243, 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.250 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.254 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.111785795, 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:49.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:49.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:49.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 124.11175398500001, 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:49.259 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:49.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.725090869, 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.268 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.715161889, 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:49.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:49.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:49.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:49.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.522827175, 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.274 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:49.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:49.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:49.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:49.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.522569754, 2024-09-17T21:49:49.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:49.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.52253690800001, 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.286 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:49.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:49.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:49.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.291 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.52251996299999, 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.293 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:49.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 101.522478466, 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:49.300 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:49.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 55.528241962999999, 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:49.303 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.304 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.305 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:49.307 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:49.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:49.309 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:49.306+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:49.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:49.409 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:49.407+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:49.458 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:49.458 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:49.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:49.468 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:49.474 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:49.474 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:49.493 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:49.510 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:49.510 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:49.511 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:49.508+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:49.527 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:49.545 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:49.545 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:49.562 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:49.563 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:49.578 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:49.578 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:49.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:49.628 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:49.626+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:49.712 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:49.709+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:49.733 INFO:teuthology.orchestra.run.smithi007.stderr:set 3.12 pg_upmap mapping to [1,3] 2024-09-17T21:49:49.742 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:49.742 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:49.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:49.813 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:49.810+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:49.908 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:49.908 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:49.913 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:49.911+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:49.917 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:50.014 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:50.012+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:50.024 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.024 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:50.115 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:50.112+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:50.139 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.139 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:50.215 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:50.213+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:50.263 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.263 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.272 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:50.316 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:50.313+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:50.380 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.380 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:50.432 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:50.430+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:50.495 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.496 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:50.518 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:50.515+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:50.619 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:50.617+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:50.652 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.652 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:50.719 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:50.717+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:50.774 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.774 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:49:50.783 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:50.826 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:50.824+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:50.896 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:50.896 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:50.905 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:50.947 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:50.944+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:51.021 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:51.019+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:51.052 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.052 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:51.123 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:51.120+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:51.168 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.168 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.177 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:51.223 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:51.221+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:51.324 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.324 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.324 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:51.322+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:51.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:51.425 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:51.423+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:51.442 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.443 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:51.526 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:51.524+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:51.559 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.559 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:51.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:51.653 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:51.651+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:51.676 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:51.676 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:51.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:51.727 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:51.724+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:51.735 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:51.735 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:51.747 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:49:51.763 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:51.763 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:51.788 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:51.788 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:51.811 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:51.811 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:51.828 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:51.825+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:51.834 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:51.856 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:51.857 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:51.865 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:51.865 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:51.865 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:51.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:51.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:51.929 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:51.926+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:52.024 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:52.025 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:52.025 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:52.025 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:52.025 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:52.030 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:52.028+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:52.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:52.131 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:52.129+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:52.182 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:52.182 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:52.182 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09588709499999, 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.183 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.185 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.186 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:52.187 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:52.188 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.094728762, 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.189 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.190 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.191 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:52.192 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:52.193 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09471221699999, 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:52.194 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.195 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.196 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:52.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09469745200001, 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.199 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:52.201 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.202 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.094670147, 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.205 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.206 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09463325900001, 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.208 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:52.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.211 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09459456099999, 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:52.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.214 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.216 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.094556388, 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.218 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.219 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:52.221 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.094268048, 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.223 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.224 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:52.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.226 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 114.09424145200001, 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:52.228 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:52.229 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.230 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:52.231 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.232 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.519703646, 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:52.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.236 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:52.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.519670331, 2024-09-17T21:49:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.241 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:52.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.243 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.244 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:52.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.512580267, 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.246 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.248 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:52.250 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:52.251 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 108.512546587, 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.252 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:52.254 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.255 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:52.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.424077656999998, 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.257 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:52.258 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.259 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:52.260 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.261 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:52.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.263 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.423984637999993, 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.264 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.265 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:52.266 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.267 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.268 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:52.269 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.270 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.421354691000005, 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.272 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.273 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.274 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:52.275 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.276 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.277 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:52.278 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 82.077811987999993, 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:52.280 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:52.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.283 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.284 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.422492491, 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:52.286 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.288 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:52.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:52.290 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.291 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.292 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 81.416583461000002, 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:52.294 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.295 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:52.296 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.297 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.299 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.299 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.299 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:52.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:52.301 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:52.298+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:52.401 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:52.399+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:52.408 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:52.408 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:52.408 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:52.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:52.502 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:52.500+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:52.575 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:52.603 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:52.601+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:52.704 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:52.702+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:52.724 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:52.724 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:52.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:52.724 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 197.84847606100001, 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:52.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:52.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:52.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.812029751, 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.731 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:52.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:52.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.81200844999999, 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.736 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:52.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.739 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.740 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.81198768900001, 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:52.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:52.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.811966583, 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.746 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:52.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.81194609400001, 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.750 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:52.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.754 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.811921997, 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.755 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:52.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.811899504, 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:52.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:52.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.811871897, 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:52.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:52.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:52.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 125.81184705699999, 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.768 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:52.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:52.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.659893282, 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.773 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.65986952, 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:52.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.659827131, 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:52.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.60407847800001, 2024-09-17T21:49:52.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:52.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:52.790 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.604055383, 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.791 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.795 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.60402900299999, 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:52.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.604012367, 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.800 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.804 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.603982403, 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:52.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:52.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.603969478, 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:52.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 121.603959257, 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:52.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:52.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:52.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:52.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:52.821 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:52.824 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:52.819+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:52.825 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:52.923 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:52.920+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:52.939 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:52.939 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:52.939 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:52.939 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:52.948 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:53.023 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:53.021+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:53.054 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:53.055 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:53.055 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.064 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:53.124 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:53.122+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 207.47022517100001, 2024-09-17T21:49:53.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:49:53.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 207.46918071100001, 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:49:53.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:49:53.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.223 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 146.26799538, 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:53.224 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:49:53.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:49:53.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 115.052077712, 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:49:53.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.235 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84964097, 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:49:53.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:49:53.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:53.242 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84958379, 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:49:53.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:49:53.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:49:53.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84901320900001, 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:49:53.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:49:53.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.848985275, 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:49:53.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:49:53.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.848850923, 2024-09-17T21:49:53.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:49:53.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:53.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.848829388, 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:49:53.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:49:53.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84866842300001, 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:49:53.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:49:53.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84861265799999, 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:49:53.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:49:53.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84726605100001, 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:49:53.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.847214495, 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:49:53.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:49:53.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.846526234, 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.293 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:49:53.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.84650918299999, 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.299 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:49:53.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:49:53.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.845570943, 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:49:53.304 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:49:53.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 114.845540603, 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:53.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:49:53.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 109.55114315500001, 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:49:53.316 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:49:53.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 108.65002545500001, 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:49:53.322 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:49:53.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:49:53.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.327 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:53.329 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:53.326+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:53.430 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:53.427+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:53.477 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.477 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:53.477 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:53.477 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.485 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:53.530 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:53.528+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:53.592 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:53.592 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:53.592 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:53.631 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:53.628+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:53.718 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:53.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74423701800001, 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:49:53.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:49:53.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.744212758, 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:49:53.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.744186875, 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.728 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:53.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:53.732 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74416187400001, 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:49:53.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74412207500001, 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:49:53.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74410632999999, 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.742 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:49:53.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.744094795, 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:49:53.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74408449500001, 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:49:53.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:49:53.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.742113252, 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:49:53.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:49:53.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.742100104, 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:49:53.760 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:49:53.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74208949800001, 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.765 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:49:53.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:49:53.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:53.769 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74207916, 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:53.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:49:53.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74206915299999, 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.774 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:49:53.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:49:53.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74205920200001, 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:49:53.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:49:53.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.783 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74204389400001, 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:49:53.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:49:53.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74005705100001, 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.788 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:49:53.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:49:53.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74004456200001, 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:49:53.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:49:53.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:53.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 216.74003409900001, 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:49:53.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.040698301000006, 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.802 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:49:53.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:53.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:53.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.685968324999997, 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:53.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:53.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:53.812 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:49:53.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:53.815 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:53.811+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:53.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:53.914 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:53.912+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:53.962 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:53.963 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:53.963 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:53.963 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:53.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:53.996 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:53.996 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:54.013 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:54.013 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:54.015 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:54.013+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:54.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:54.030 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:54.047 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:54.047 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:54.063 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:54.064 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:54.079 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.079 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.080 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:54.080 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:54.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:54.132 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:54.130+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:54.217 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:54.215+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:54.236 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.236 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:54.317 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:54.315+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:54.353 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.353 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:54.418 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:54.416+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:54.469 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.469 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:54.519 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:54.516+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:54.620 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:54.618+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:54.625 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.625 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:54.721 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:54.719+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:54.741 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.741 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:49:54.758 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:49:54.759 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:49:54.759 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-17T21:49:54.759 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-17T21:49:54.759 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-17T21:49:54.854 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:54.852+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:54.871 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:54.871 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:54.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:54.923 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:54.921+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:55.023 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:55.021+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:55.031 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.031 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.040 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:55.124 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:55.122+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:55.147 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.147 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:55.225 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:55.223+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:55.262 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.263 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:55.325 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:55.323+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:55.378 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.378 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.388 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:55.433 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:55.431+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:55.509 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.509 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:55.526 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:55.524+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:55.627 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:55.625+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:55.667 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.667 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.676 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:55.728 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:55.726+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:55.754 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-17T21:49:55.829 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.829 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.829 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:55.827+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:55.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:55.930 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:55.927+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:55.944 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:55.944 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:55.953 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:56.033 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:56.031+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:56.059 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:56.059 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:56.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:56.092 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:49:56.135 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:56.133+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:56.175 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:56.175 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:56.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:56.222 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:56.222 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:56.234 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:56.232+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:56.239 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:56.239 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:56.255 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:56.255 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:56.272 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:56.272 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:56.288 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:56.289 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:56.292 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:56.292 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:56.292 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:56.301 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:56.305 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:56.305 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:56.343 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:56.340+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:56.436 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:56.434+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:56.449 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:56.449 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:56.450 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:56.458 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:56.537 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:56.535+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:56.565 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:56.565 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590538675000005, 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.566 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:49:56.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:56.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590514537999994, 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:56.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:49:56.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590482283, 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.575 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:56.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:49:56.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590452229999997, 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590428884000005, 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:49:56.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590375937000005, 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:49:56.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:49:56.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:49:56.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.590342590999995, 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:49:56.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:49:56.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.587011357999998, 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.604 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586983458000006, 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:49:56.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586960751999996, 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:49:56.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:49:56.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586944484, 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:49:56.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:49:56.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586934513000003, 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:49:56.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:49:56.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586922364000003, 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:49:56.623 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:49:56.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586912669, 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586903075999999, 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:49:56.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:49:56.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586893586000002, 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:49:56.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:49:56.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586883740000005, 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:49:56.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.586874399999999, 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:49:56.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:49:56.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.888660986000005, 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:49:56.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:49:56.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:56.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.888071014000005, 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:49:56.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:49:56.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:49:56.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:49:56.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:49:56.662 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:56.663 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:56.660+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:56.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:56.764 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:56.761+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:56.771 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:49:56.782 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:56.782 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:56.783 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:56.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:56.865 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:56.862+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:56.945 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:56.945 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:56.945 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:56.945 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:56.946 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:56.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:56.966 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:56.963+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 215.378466242, 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:49:57.065 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.067 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:49:57.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.070 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:57.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 215.34619126699999, 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:49:57.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 215.34595249899999, 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:49:57.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 166.178446369, 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:49:57.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:49:57.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:49:57.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 166.171544598, 2024-09-17T21:49:57.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:57.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.090 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:49:57.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 166.171428507, 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:49:57.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:49:57.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:49:57.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.97442498699999, 2024-09-17T21:49:57.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.974386192, 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:49:57.104 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:49:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.107 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.97437190799999, 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:49:57.109 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:49:57.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.974126942, 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:49:57.114 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:49:57.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.974044494, 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:49:57.119 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:49:57.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.974012684, 2024-09-17T21:49:57.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:49:57.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.58734956799999, 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:49:57.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:49:57.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.133 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.134 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.577420588, 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:57.135 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:49:57.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:49:57.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:57.140 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.385085874, 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:57.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:49:57.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:49:57.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.384828453, 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.147 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.150 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.384795607, 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:49:57.153 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:49:57.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:49:57.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.384778662, 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:49:57.159 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.161 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:49:57.163 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.164 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:49:57.165 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 109.384737165, 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.166 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:49:57.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.168 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:49:57.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.170 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 63.390500662000001, 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:49:57.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.173 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.174 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.175 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:49:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.177 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.178 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:57.179 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:57.176+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:57.279 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:57.277+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:57.286 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:57.287 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:49:57.381 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:57.379+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:57.403 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.403 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:57.403 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:57.403 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:57.403 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:49:57.481 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:57.479+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.521 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.434401525, 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:49:57.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:49:57.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.43324319200001, 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.527 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:57.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:49:57.531 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.433226647, 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.535 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.536 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.43321188199999, 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.537 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:49:57.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.43318457700001, 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.542 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.544 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:57.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.43314768899999, 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:49:57.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:49:57.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.433108991, 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.552 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:49:57.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.433070818, 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:49:57.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:49:57.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.43278247800001, 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.562 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.563 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:49:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:49:57.565 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.566 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 119.432755882, 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.567 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.568 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:49:57.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:49:57.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.858218076, 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:49:57.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:49:57.574 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:49:57.575 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.576 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:57.577 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.858184761, 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.578 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:49:57.579 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.580 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.581 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:49:57.582 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.583 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.85109469699999, 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:49:57.584 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.585 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:49:57.586 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.587 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.588 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 113.85106101700001, 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.589 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.590 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:49:57.591 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:49:57.592 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:49:57.593 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:57.594 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.762592087000002, 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.595 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.596 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.597 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:49:57.598 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:49:57.599 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.600 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.762499067999997, 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.601 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:57.602 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:49:57.603 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.604 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:49:57.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.759869120999994, 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:49:57.609 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.610 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.611 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:49:57.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 87.416326417999997, 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.761006921000003, 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.623 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:49:57.624 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.626 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 86.755097891000005, 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:49:57.628 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:57.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:49:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:49:57.631 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:49:57.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:49:57.633 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:57.634 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:49:57.636 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:57.633+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:57.736 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:57.734+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:57.758 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:49:57.791 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.791 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:57.791 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:49:57.791 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:49:57.841 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:57.839+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:57.938 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:57.935+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:57.953 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:49:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:49:57.953 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:49:57.953 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:57.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:49:58.038 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:58.036+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:58.070 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:49:58.070 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:49:58.070 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:49:58.070 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 203.19430056799999, 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:49:58.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:49:58.073 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:49:58.074 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:49:58.075 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15785425799999, 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.077 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:49:58.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:49:58.079 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.080 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15783295700001, 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:49:58.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:49:58.084 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:58.085 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15781219600001, 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.086 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:49:58.087 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.088 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.089 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15779108999999, 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.090 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:49:58.091 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:49:58.092 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:49:58.093 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15777060100001, 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.094 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.095 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:49:58.096 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:49:58.097 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15774650399999, 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:49:58.098 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.099 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.100 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:49:58.101 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.102 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.157724011, 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.103 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.104 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.105 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.106 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.15769640400001, 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:49:58.107 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.108 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.109 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.110 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 131.157671564, 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.111 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:58.112 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:49:58.113 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:49:58.114 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:58.115 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.005717789, 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.116 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.117 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.118 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.119 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.005694027, 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:49:58.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:49:58.121 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.122 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.123 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.005651638, 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.124 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:49:58.125 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:49:58.126 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:49:58.127 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.94990298499999, 2024-09-17T21:49:58.128 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:49:58.130 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:49:58.131 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.94987989000001, 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.133 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:49:58.134 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:49:58.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:49:58.136 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.94985351, 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.137 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:49:58.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.949836874, 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.142 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.143 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.144 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:58.145 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.94980691000001, 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:58.146 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:49:58.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:49:58.148 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.949793985, 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.150 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:49:58.151 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.152 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 126.949783764, 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:49:58.154 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:49:58.155 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:49:58.156 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:49:58.157 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:49:58.158 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:49:58.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:49:58.160 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:58.157+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:58.260 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:58.258+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:58.271 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:58.272 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:58.272 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:49:58.272 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:58.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:49:58.317 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:49:58.361 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:58.359+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:58.428 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:49:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:49:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:49:58.428 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:49:58.428 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:49:58.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:49:58.442 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:49:58.443 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:49:58.460 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:49:58.460 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:49:58.462 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:58.460+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:58.476 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:49:58.476 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:49:58.493 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:49:58.494 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:49:58.510 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:49:58.510 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:49:58.528 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:49:58.528 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:49:58.548 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:58.548 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:58.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:49:58.599 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:58.596+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:58.662 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:58.662 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:58.664 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:58.662+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:58.671 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:49:58.765 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:49:58.762+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:49:58.828 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:58.828 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:58.838 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:49:58.866 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:58.864+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:58.966 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:58.964+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:58.984 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:58.984 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:58.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:49:59.067 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:59.065+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:59.101 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.101 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:49:59.193 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:59.191+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.257 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.266 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:49:59.307 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:49:59.304+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:49:59.370 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:59.368+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:49:59.413 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.413 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.422 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:49:59.471 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:59.469+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:59.542 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.543 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.552 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:49:59.573 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:49:59.570+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:49:59.673 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:59.671+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:59.702 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.702 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:49:59.779 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-17T21:49:59.795 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:49:59.793+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:49:59.873 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:49:59.873 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:49:59.877 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:49:59.875+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:49:59.884 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:49:59.978 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:49:59.976+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:00.033 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:00.033 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:00.041 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:50:00.082 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:00.080+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:00.180 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:00.177+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:00.190 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:00.190 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:00.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:50:00.280 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:00.278+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:00.306 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:00.306 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:00.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:50:00.399 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:00.397+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:00.464 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:00.464 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:00.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:50:00.514 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:00.512+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:00.540 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:50:00.583 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:00.581+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:00.621 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:00.621 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:00.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:50:00.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:50:00.681 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:50:00.684 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:00.682+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:00.699 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:50:00.699 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:50:00.716 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:50:00.716 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:50:00.734 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:50:00.734 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:50:00.741 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 205.86522135199999, 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:50:00.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:50:00.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82877504199999, 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:50:00.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:00.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82875374100001, 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:50:00.753 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:50:00.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:50:00.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82873298000001, 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.758 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:50:00.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:50:00.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:00.762 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82871187399999, 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:50:00.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82869138500001, 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:50:00.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.770 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82866728799999, 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:00.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.775 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.828644795, 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:50:00.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.82861718800001, 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.780 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:50:00.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.784 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.828592348, 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:50:00.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.67663857299999, 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:50:00.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.793 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.67661481100001, 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.794 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.795 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.796 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.797 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.67657242199999, 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.798 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.799 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:50:00.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.802 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.620823769, 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:50:00.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.804 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:50:00.805 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:50:00.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.62080067400001, 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.808 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:50:00.810 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.811 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.620774294, 2024-09-17T21:50:00.812 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.814 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:50:00.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.620757658, 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.817 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:50:00.819 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.820 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.821 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.62072769400001, 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.822 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.823 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:50:00.825 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.62071476899999, 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:50:00.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:50:00.829 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 129.62070454799999, 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:50:00.830 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:50:00.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:00.833 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:00.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:00.835 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:00.835 INFO:tasks.thrashosds.thrasher:in_osds: [1, 5, 4, 0, 2, 3] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 4, 5, 3] 2024-09-17T21:50:00.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:50:00.837 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:50:00.837 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:50:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:50:00.838 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:50:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:50:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is set 2024-09-17T21:50:00.840 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:00.834+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:00.937 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:00.935+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:00.961 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:00.961 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:00.961 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:00.961 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:00.970 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:50:01.038 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:01.036+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:01.118 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.127 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:50:01.140 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:01.137+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:01.236 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.493926456, 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:50:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:50:01.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 215.49288199599999, 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:50:01.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:50:01.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:50:01.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 154.29169666499999, 2024-09-17T21:50:01.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:50:01.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:50:01.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 123.075778997, 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:50:01.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.259 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.873342255, 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:50:01.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.873285075, 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.267 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:50:01.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:50:01.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:01.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.273 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87271449399999, 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:50:01.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:50:01.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:50:01.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87268656000001, 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.278 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:50:01.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.872552208, 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:50:01.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.288 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.872530673, 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:50:01.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:01.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87236970799999, 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:50:01.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:50:01.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87231394299999, 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:50:01.299 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:50:01.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:01.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87096733600001, 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:50:01.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.87091578, 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:50:01.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.870227519, 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:50:01.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:01.320 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.870210468, 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.325 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.869272228, 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:01.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:01.330 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.869241888, 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:01.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.335 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 117.57484444000001, 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:50:01.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:50:01.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:01.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 116.67372674000001, 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.346 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.348 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:50:01.348 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:01.345+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:01.448 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:01.445+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:01.456 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:01.456 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:01.456 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:50:01.549 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:01.547+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:01.612 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.613 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:01.613 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:01.613 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:01.613 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:50:01.663 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:01.660+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:01.751 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:01.748+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79618845799999, 2024-09-17T21:50:01.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:50:01.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79616419800001, 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.775 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79613831500001, 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:50:01.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:50:01.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.796113314, 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:01.789 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79607351499999, 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:50:01.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79605777, 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:50:01.794 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:50:01.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:50:01.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79604623500001, 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:01.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79603593499999, 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:50:01.803 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:01.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:50:01.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79406469200001, 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:50:01.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:50:01.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79405154400001, 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:50:01.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79404093799999, 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:01.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:50:01.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79403060000001, 2024-09-17T21:50:01.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.794020593, 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:50:01.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79401064199999, 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:50:01.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:50:01.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79399533399999, 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:50:01.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79200849099999, 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:01.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.79199600199999, 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:50:01.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 224.791985539, 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:50:01.850 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:50:01.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:50:01.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.092649741000002, 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:01.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 63.737919765000001, 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:01.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:01.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:01.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:01.865 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.867 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:50:01.867 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:50:01.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:50:01.868 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:01.863+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:01.967 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:01.965+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:01.977 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:01.977 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:01.977 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:01.977 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:01.986 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:50:02.069 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:02.066+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:02.093 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:02.093 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:02.093 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:02.102 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:50:02.169 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:02.167+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:02.210 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:02.210 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:02.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:02.210 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:02.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235479941999998, 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.211 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:02.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235455805000001, 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:50:02.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235423549999993, 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.220 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:02.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235393497000004, 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.224 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.228 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235370150999998, 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235317203999998, 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.235283858000003, 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:02.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:02.241 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231952625000005, 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:50:02.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.245 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231924724999999, 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.249 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231902019000003, 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:50:02.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:50:02.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:50:02.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:50:02.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231885750999993, 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231875779999996, 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.258 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:50:02.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.262 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231863630999996, 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.266 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231853935999993, 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.267 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231844343000006, 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:50:02.271 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231834852999995, 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:02.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231825006999998, 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.280 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:02.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:02.284 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 87.231815667000006, 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.288 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.533602252999998, 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:50:02.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.293 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.533012280999998, 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.294 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:50:02.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:50:02.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:50:02.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.298 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:02.299 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:02.299 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:02.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:50:02.300 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:02.298+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:02.401 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:02.399+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:02.407 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:02.407 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:50:02.407 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:02.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:50:02.501 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:02.499+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:02.523 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:50:02.523 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:50:02.523 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:02.532 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:50:02.602 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:02.600+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 220.995648577, 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:50:02.682 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.684 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.685 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:50:02.687 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 220.96337360199999, 2024-09-17T21:50:02.688 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:50:02.690 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:50:02.691 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.692 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:02.693 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 220.96313483399999, 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:02.694 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:50:02.695 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.696 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:02.698 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.79562870399999, 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.699 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:50:02.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:50:02.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:50:02.702 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.703 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:02.704 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.78872693299999, 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.706 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.707 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.708 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:02.709 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 171.788610842, 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:02.710 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.711 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:50:02.713 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.59160732199999, 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:50:02.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:50:02.718 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.59156852699999, 2024-09-17T21:50:02.719 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:50:02.721 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.722 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:50:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.59155424299999, 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:50:02.724 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:50:02.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.727 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.728 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.59130927699999, 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.729 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:02.730 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:50:02.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:50:02.733 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.59122682899999, 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.734 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:02.735 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:50:02.736 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.737 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 137.591195019, 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:02.739 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:50:02.741 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:50:02.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.204531903, 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.744 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.745 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.746 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:50:02.747 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.748 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:50:02.749 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:02.750 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 133.19460292299999, 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.751 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.752 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.753 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.754 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:50:02.755 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:02.756 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.00226820899999, 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.757 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:50:02.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.759 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:50:02.761 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.762 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.763 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.00201078800001, 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:50:02.765 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.766 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:50:02.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:50:02.768 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:50:02.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.001977942, 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:02.770 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:50:02.772 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.773 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.774 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.001960997, 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:02.776 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.777 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:50:02.778 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:50:02.779 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.781 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 115.0019195, 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:02.782 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.783 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:50:02.784 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.785 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.786 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:50:02.787 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 69.007682997000003, 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:50:02.788 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.789 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.790 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.791 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.792 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:02.793 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:02.793 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:02.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:50:02.795 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:50:02.795 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:02.792+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:02.803 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:50:02.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:50:02.840 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-17T21:50:02.895 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:02.892+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:02.966 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:02.966 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:02.966 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:50:02.966 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:02.977 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:50:02.993 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:50:02.994 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:50:02.995 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:02.993+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:03.011 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:50:03.011 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:50:03.030 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:50:03.030 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:50:03.047 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:50:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:50:03.064 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:50:03.065 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:50:03.096 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:03.093+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:03.099 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:50:03.099 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:50:03.104 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.104 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:50:03.198 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:03.196+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:03.261 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.261 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:50:03.311 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:03.309+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:03.398 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:03.396+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.418 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:50:03.499 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:03.497+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:03.534 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.534 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:50:03.600 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:03.597+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:03.651 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.651 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:50:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:50:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is set 2024-09-17T21:50:03.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:50:03.735 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:03.733+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:03.800 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:03.798+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:03.801 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.801 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.811 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:50:03.901 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:03.898+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:03.959 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:03.959 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:03.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:50:04.008 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:04.006+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:04.102 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:04.100+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:04.115 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.115 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.124 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:50:04.203 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:04.201+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:04.232 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.232 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:50:04.305 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:04.303+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:04.348 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.348 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.357 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:50:04.406 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:04.403+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:04.464 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.464 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.473 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:50:04.514 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:04.512+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:04.607 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:04.604+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:04.620 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:50:04.708 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:04.706+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:04.776 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.777 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:50:04.792 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:50:04.810 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:50:04.810 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:04.807+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:04.910 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:04.908+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:04.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:50:05.011 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:05.009+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:05.050 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:05.050 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:05.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:50:05.112 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:50:05.113 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:05.111+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:05.208 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:05.213 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:05.211+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:05.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:50:05.267 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:50:05.268 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:50:05.285 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:50:05.285 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:50:05.304 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:50:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:50:05.314 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:05.312+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:05.320 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:50:05.320 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:50:05.330 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:05.330 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:50:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:50:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:50:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:306 3.1 3:8018393d:::rbd_header.11125c054388:head [watch reconnect cookie 140701653807984 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:05.331 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 223.64269533000001, 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008506991, 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.332 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 306 2024-09-17T21:50:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.333 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682821+0000", 2024-09-17T21:50:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682823+0000", 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5e-06 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.336 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682828+0000", 2024-09-17T21:50:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.694e-06 2024-09-17T21:50:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.337 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682841+0000", 2024-09-17T21:50:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2461e-05 2024-09-17T21:50:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.338 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682849+0000", 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4179999999999997e-06 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.339 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682923+0000", 2024-09-17T21:50:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3813999999999995e-05 2024-09-17T21:50:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:05.340 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.682931+0000", 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6790000000000005e-06 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685879+0000", 2024-09-17T21:50:05.341 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002948506 2024-09-17T21:50:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.685891+0000", 2024-09-17T21:50:05.342 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1501999999999999e-05 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689381+0000", 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.003490392 2024-09-17T21:50:05.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689565+0000", 2024-09-17T21:50:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018402300000000001 2024-09-17T21:50:05.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.691328+0000", 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0017630020000000001 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:390 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [read 2097152~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:05.346 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 223.610420355, 2024-09-17T21:50:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97451681199999995, 2024-09-17T21:50:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.347 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 390 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.349 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715096+0000", 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.350 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715098+0000", 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.728e-06 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715105+0000", 2024-09-17T21:50:05.351 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4279999999999999e-06 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715138+0000", 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2419000000000003e-05 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715145+0000", 2024-09-17T21:50:05.352 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5349999999999999e-06 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715214+0000", 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9047999999999993e-05 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715226+0000", 2024-09-17T21:50:05.353 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1731e-05 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689222+0000", 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.973995846 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689472+0000", 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024968400000000002 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.354 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689613+0000", 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014139299999999999 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4648.0:391 3.1b 3:daacb18f:::rbd_data.11125c054388.0000000000000006:head [sparse-read 2101248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e39)", 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 223.610181587, 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97475718700000002, 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.355 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4648", 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/481752346", 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 391 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.356 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715335+0000", 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715336+0000", 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4499999999999995e-07 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.357 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715341+0000", 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.215e-06 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715353+0000", 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2048000000000001e-05 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.358 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715361+0000", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5800000000000003e-06 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715401+0000", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9920999999999999e-05 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:21.715407+0000", 2024-09-17T21:50:05.359 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3840000000000002e-06 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689936+0000", 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.97452878499999995 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.689996+0000", 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9877000000000003e-05 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:22.690092+0000", 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6432000000000001e-05 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:12040 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140714605036160 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 174.44267545700001, 2024-09-17T21:50:05.361 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010980185, 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12040 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882841+0000", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882843+0000", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882852+0000", 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0210000000000005e-06 2024-09-17T21:50:05.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882875+0000", 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2840000000000002e-05 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882887+0000", 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2150000000000001e-05 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.364 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882980+0000", 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2816e-05 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.882991+0000", 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1028e-05 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886864+0000", 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0038733880000000002 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.886876+0000", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2131e-05 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.890926+0000", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0040495549999999 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891095+0000", 2024-09-17T21:50:05.366 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000169302 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.893821+0000", 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.002726034 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2163 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [watch reconnect cookie 140296314743056 gen 6] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 174.435773686, 2024-09-17T21:50:05.367 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.004625163, 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2163 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889743+0000", 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889744+0000", 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.251e-06 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.369 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889750+0000", 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9290000000000003e-06 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889759+0000", 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1649999999999995e-06 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889766+0000", 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2779999999999996e-06 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889823+0000", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6125999999999998e-05 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889833+0000", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9690000000000006e-06 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891905+0000", 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0020726 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.891996+0000", 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.0878999999999995e-05 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.894368+0000", 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0023719660000000001 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5051.0:2164 3.1 3:823e0bba:::rbd_header.1112f0bef54a:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 174.43565759500001, 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0028292480000001, 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5051", 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/651141807", 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2164 2024-09-17T21:50:05.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889859+0000", 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.374 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889860+0000", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2300000000000001e-07 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889864+0000", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3970000000000004e-06 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889870+0000", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9279999999999998e-06 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.375 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889874+0000", 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8179999999999997e-06 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889920+0000", 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6579000000000002e-05 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:10.889926+0000", 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0460000000000002e-06 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.376 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892438+0000", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0025120590000001 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892536+0000", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7386999999999994e-05 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:11.892688+0000", 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000152411 2024-09-17T21:50:05.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12770 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.238654075, 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800607899999997, 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.378 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12770 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086862+0000", 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.379 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086863+0000", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086864+0000", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5999999999999999e-06 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086868+0000", 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4970000000000001e-06 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.380 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086871+0000", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7580000000000001e-06 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086966+0000", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4986000000000002e-05 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086971+0000", 2024-09-17T21:50:05.381 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9710000000000003e-06 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004858+0000", 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91788735899999996 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004869+0000", 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0543000000000001e-05 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.382 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12771 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.23861528, 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91798457600000005, 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.383 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12771 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086901+0000", 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.384 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086902+0000", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6600000000000002e-07 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086904+0000", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.278e-06 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086908+0000", 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1570000000000002e-06 2024-09-17T21:50:05.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086911+0000", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8399999999999999e-06 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087008+0000", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7356999999999996e-05 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087015+0000", 2024-09-17T21:50:05.386 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6540000000000002e-06 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004880+0000", 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91786525399999996 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004886+0000", 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6699999999999999e-06 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12772 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.238600996, 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91800588599999999, 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.388 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12772 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086916+0000", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.36e-07 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086917+0000", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1930000000000001e-06 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086920+0000", 2024-09-17T21:50:05.390 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1700000000000001e-06 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.086923+0000", 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.903e-06 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087051+0000", 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012762200000000001 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.391 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087057+0000", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6100000000000002e-06 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004912+0000", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91785491600000002 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004921+0000", 2024-09-17T21:50:05.392 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2359999999999993e-06 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12789 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.23835603000001, 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91777569000000003, 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.393 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12789 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:05.394 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087161+0000", 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0699999999999998e-07 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087163+0000", 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5500000000000001e-06 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.395 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087169+0000", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.062e-06 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087173+0000", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8829999999999997e-06 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087217+0000", 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:50:05.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087225+0000", 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3769999999999998e-06 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004932+0000", 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91770757800000002 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.397 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004936+0000", 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0799999999999999e-06 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12790 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.23827358200001, 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91771613399999996, 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.398 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12790 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.399 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087243+0000", 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087244+0000", 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5700000000000002e-07 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087247+0000", 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.18e-06 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.400 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087254+0000", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5009999999999998e-06 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087261+0000", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.2910000000000004e-06 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087303+0000", 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2271000000000002e-05 2024-09-17T21:50:05.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087309+0000", 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4019999999999999e-06 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004951+0000", 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91764142999999998 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.402 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004959+0000", 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4020000000000002e-06 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5378.0:12791 3.1b 3.1aa907bb (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e110)", 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 140.23824177200001, 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91769709600000005, 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.403 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5378", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1329900253", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 12791 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.404 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087275+0000", 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1799999999999995e-07 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.405 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087278+0000", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4030000000000001e-06 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087285+0000", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.379e-06 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087291+0000", 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087337+0000", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6310999999999998e-05 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:45.087343+0000", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4150000000000004e-06 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004968+0000", 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.91762427000000002 2024-09-17T21:50:05.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.004972+0000", 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3000000000000003e-06 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20904 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 0~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.85157865599999, 2024-09-17T21:50:05.408 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.550416222, 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20904 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473938+0000", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473940+0000", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9929999999999998e-06 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473949+0000", 2024-09-17T21:50:05.410 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4399999999999994e-06 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473977+0000", 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8062999999999999e-05 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.473993+0000", 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5223e-05 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.411 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474077+0000", 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3935000000000002e-05 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.474083+0000", 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8340000000000001e-06 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012375+0000", 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.53829166299999998 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012389+0000", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4124000000000001e-05 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024204+0000", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.011814717 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024354+0000", 2024-09-17T21:50:05.413 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00015023 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:20960 3.6 3:64b77586:::rbd_data.1112ffdab28d.000000000000004e:head [read 26624~26624] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e112)", 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.841649676, 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.54077708, 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:05.414 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 20960 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483867+0000", 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.415 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483868+0000", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0299999999999998e-07 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483872+0000", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9770000000000002e-06 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483882+0000", 2024-09-17T21:50:05.416 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.012e-05 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483887+0000", 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0810000000000001e-06 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483930+0000", 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3689000000000001e-05 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.417 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:49.483936+0000", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0560000000000001e-06 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012443+0000", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.52850671400000004 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:50.012454+0000", 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0869999999999999e-05 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.418 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024526+0000", 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0120718929999999 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.024644+0000", 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000117977 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.419 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:226 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 524288~21504] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.64931496200001, 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016807437, 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 226 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.420 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676202+0000", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676203+0000", 2024-09-17T21:50:05.421 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9120000000000001e-06 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676303+0000", 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9108e-05 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676314+0000", 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1762e-05 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.422 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676324+0000", 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8090000000000004e-06 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0001752 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1977999999999999e-05 2024-09-17T21:50:05.423 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676595+0000", 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.000083925 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.676948+0000", 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00035278700000000002 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 4", 2024-09-17T21:50:05.424 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.682702+0000", 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0057534969999999998 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.687194+0000", 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044929280000000002 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692872+0000", 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0056770789999999998 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.425 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.692910+0000", 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8624000000000002e-05 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693009+0000", 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8827999999999996e-05 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:262 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 262144~32768] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:05.426 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.649057541, 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0167097970000001, 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 262 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.427 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676459+0000", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676460+0000", 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7100000000000001e-07 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.428 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676464+0000", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.9809999999999997e-06 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676474+0000", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.9790000000000005e-06 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676481+0000", 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4660000000000002e-06 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.429 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676548+0000", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6502999999999997e-05 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676557+0000", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5589999999999994e-06 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686553+0000", 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0099955679999999 2024-09-17T21:50:05.430 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686631+0000", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8434000000000002e-05 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693046+0000", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064146000000000003 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693092+0000", 2024-09-17T21:50:05.431 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6721000000000002e-05 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693169+0000", 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6315000000000004e-05 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:265 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 294912~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.64902469499999, 2024-09-17T21:50:05.432 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.016835191, 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 265 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:05.433 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676492+0000", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676494+0000", 2024-09-17T21:50:05.434 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1299999999999999e-06 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676499+0000", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2529999999999998e-06 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676502+0000", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5719999999999999e-06 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676597+0000", 2024-09-17T21:50:05.435 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4919000000000004e-05 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676605+0000", 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2400000000000007e-06 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686660+0000", 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0100549999999999 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:05.436 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686728+0000", 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7904000000000006e-05 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693207+0000", 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0064790459999999996 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693249+0000", 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1671999999999997e-05 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.437 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693327+0000", 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8095000000000004e-05 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:266 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 311296~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.64900775, 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0170639610000001, 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.438 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 266 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:05.439 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676509+0000", 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5900000000000003e-07 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.440 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676511+0000", 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1100000000000001e-06 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676517+0000", 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.156e-06 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676523+0000", 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.13e-06 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.441 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676637+0000", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011485900000000001 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676645+0000", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0299999999999994e-06 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686757+0000", 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0101117049999999 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.442 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686827+0000", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9451999999999998e-05 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693388+0000", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065614480000000001 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693464+0000", 2024-09-17T21:50:05.443 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6107999999999997e-05 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693573+0000", 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010860400000000001 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5809.0:268 3.1b 3:dcd7c67e:::rbd_data.1112cb660a00.000000000000005e:head [read 307200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e129)", 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 117.648966253, 2024-09-17T21:50:05.444 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.017275857, 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5809", 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/3239005921", 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 268 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.445 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676550+0000", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676551+0000", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1600000000000001e-07 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676554+0000", 2024-09-17T21:50:05.446 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.095e-06 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676562+0000", 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.3089999999999996e-06 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676568+0000", 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7209999999999999e-06 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.447 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676671+0000", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103315 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:07.676679+0000", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2490000000000001e-06 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686864+0000", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.010184438 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:05.448 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.686930+0000", 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5908999999999995e-05 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693614+0000", 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0066837110000000002 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693718+0000", 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000104314 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.449 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:08.693826+0000", 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010828 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6474.0:4108 3.12 3:4e22bbfe:::rbd_data.111298c81d.000000000000001c:38 [read 1687552~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171)", 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 71.654729750000001, 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0031280950000001, 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.450 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6474", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2232973486", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 4108 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670787+0000", 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670788+0000", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.381e-06 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670794+0000", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1319999999999999e-06 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670813+0000", 2024-09-17T21:50:05.452 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8629000000000001e-05 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670825+0000", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1781e-05 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670889+0000", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4011999999999996e-05 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:53.670900+0000", 2024-09-17T21:50:05.453 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1e-05 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673538+0000", 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.00263813 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673687+0000", 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00014919400000000001 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.454 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:54.673915+0000", 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00022783600000000001 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.455 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:05.456 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:50:05.456 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:50:05.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:50:05.457 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:50:05.458 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:50:05.458 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:05.454+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:05.558 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:05.556+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:05.605 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:05.605 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:50:05.605 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:05.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:50:05.658 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:05.656+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:05.695 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:50:05.721 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:05.721 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:05.721 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:50:05.722 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:50:05.722 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:05.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:50:05.772 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:05.770+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:05.790 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:50:05.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:50:05.859 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:05.857+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13805 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 3467264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.809813718, 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99018540600000005, 2024-09-17T21:50:05.898 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13805 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083044+0000", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083045+0000", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1e-07 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083046+0000", 2024-09-17T21:50:05.900 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4810000000000001e-06 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083049+0000", 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1530000000000001e-06 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083052+0000", 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5160000000000001e-06 2024-09-17T21:50:05.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083355+0000", 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00030288500000000002 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.083361+0000", 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7660000000000001e-06 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.902 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.072964+0000", 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960282200000005 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073035+0000", 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0621e-05 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.073230+0000", 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000194952 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13837 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.80865538499999, 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013516700000004, 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13837 2024-09-17T21:50:05.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.905 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084203+0000", 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8099999999999999e-07 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084205+0000", 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.029e-06 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084210+0000", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9089999999999999e-06 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084214+0000", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.605e-06 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084691+0000", 2024-09-17T21:50:05.907 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00047646599999999999 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084701+0000", 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0318e-05 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074331+0000", 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962930299999996 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.908 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074338+0000", 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2559999999999996e-06 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13838 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.80863884, 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014642100000005, 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13838 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.910 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084219+0000", 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.911 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084220+0000", 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0800000000000001e-07 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.912 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084221+0000", 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.607e-06 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084226+0000", 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6940000000000003e-06 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084230+0000", 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8689999999999997e-06 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.913 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084725+0000", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00049573399999999995 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084729+0000", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.461e-06 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074359+0000", 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962974100000001 2024-09-17T21:50:05.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074366+0000", 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.007e-06 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13839 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:50:05.915 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.808624075, 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016003699999999, 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13839 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.916 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.917 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084234+0000", 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1899999999999998e-07 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084236+0000", 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.8300000000000001e-06 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.918 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084241+0000", 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.673e-06 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084245+0000", 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6419999999999998e-06 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.919 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084753+0000", 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00050788299999999999 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084757+0000", 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3809999999999999e-06 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.920 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074386+0000", 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962973200000004 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074394+0000", 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.5769999999999996e-06 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13840 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.80859676999999, 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99016088000000002, 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.922 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13840 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.923 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084261+0000", 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.924 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084262+0000", 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.1399999999999998e-07 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084263+0000", 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9069999999999999e-06 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084269+0000", 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084273+0000", 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0439999999999997e-06 2024-09-17T21:50:05.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084785+0000", 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051228299999999999 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084791+0000", 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:50:05.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074415+0000", 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98962401600000005 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074422+0000", 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.968e-06 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.930 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13841 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.930 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:50:05.930 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.808559882, 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99015289699999998, 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13841 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.932 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.933 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:50:05.934 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.934 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.934 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.934 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084298+0000", 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0899999999999997e-07 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084300+0000", 2024-09-17T21:50:05.935 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9530000000000002e-06 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084306+0000", 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.908e-06 2024-09-17T21:50:05.936 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084311+0000", 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2370000000000003e-06 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084829+0000", 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00051797300000000002 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.938 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.939 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084836+0000", 2024-09-17T21:50:05.939 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1690000000000003e-06 2024-09-17T21:50:05.939 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.939 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074444+0000", 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98960816600000001 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074451+0000", 2024-09-17T21:50:05.941 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.182e-06 2024-09-17T21:50:05.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.941 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13842 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.808521184, 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99014232800000002, 2024-09-17T21:50:05.942 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.943 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13842 2024-09-17T21:50:05.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.944 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.944 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.944 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.944 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:50:05.945 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084337+0000", 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.6899999999999998e-07 2024-09-17T21:50:05.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.947 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.947 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.947 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084339+0000", 2024-09-17T21:50:05.947 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.154e-06 2024-09-17T21:50:05.947 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084345+0000", 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0900000000000001e-06 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084350+0000", 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.51e-06 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.949 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.950 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084872+0000", 2024-09-17T21:50:05.950 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00052214299999999998 2024-09-17T21:50:05.950 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.950 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084880+0000", 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.498e-06 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.951 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074472+0000", 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98959249199999999 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074479+0000", 2024-09-17T21:50:05.952 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0720000000000002e-06 2024-09-17T21:50:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.954 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13843 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.954 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:50:05.954 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.80848301100001, 2024-09-17T21:50:05.954 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99013064399999995, 2024-09-17T21:50:05.954 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.955 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13843 2024-09-17T21:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.956 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:50:05.957 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.957 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.957 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:50:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084375+0000", 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.5999999999999999e-07 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.959 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.960 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084378+0000", 2024-09-17T21:50:05.960 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4990000000000001e-06 2024-09-17T21:50:05.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.960 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.961 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084385+0000", 2024-09-17T21:50:05.961 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:50:05.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.961 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084391+0000", 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4530000000000001e-06 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.962 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084927+0000", 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00053658899999999997 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084936+0000", 2024-09-17T21:50:05.963 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4009999999999997e-06 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074499+0000", 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98956343000000002 2024-09-17T21:50:05.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.965 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.965 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074506+0000", 2024-09-17T21:50:05.965 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5780000000000004e-06 2024-09-17T21:50:05.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13853 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.966 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.808194671, 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012964999999997, 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.967 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.968 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.968 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13853 2024-09-17T21:50:05.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.968 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084663+0000", 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.970 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084664+0000", 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3599999999999999e-07 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.971 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084666+0000", 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9199999999999998e-06 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084672+0000", 2024-09-17T21:50:05.972 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.127e-06 2024-09-17T21:50:05.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.973 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.973 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084677+0000", 2024-09-17T21:50:05.973 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7709999999999997e-06 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085426+0000", 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000749531 2024-09-17T21:50:05.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085430+0000", 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7780000000000001e-06 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074787+0000", 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98935681799999997 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.976 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074793+0000", 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3690000000000003e-06 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5417.0:13854 3.1e 3.f43b1dbe (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 127.808168075, 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.99012865000000005, 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "reached pg", 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5417", 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:05.978 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 13854 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.979 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084690+0000", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.65e-07 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084693+0000", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2340000000000001e-06 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084704+0000", 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1129e-05 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.084709+0000", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.485e-06 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085460+0000", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00075049399999999999 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:58.085464+0000", 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1069999999999998e-06 2024-09-17T21:50:05.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074812+0000", 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.98934842199999995 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:59.074819+0000", 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4139999999999999e-06 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17358 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 311296~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.233630269, 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008033258, 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17358 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.983 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659228+0000", 2024-09-17T21:50:05.984 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3399999999999999e-07 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659232+0000", 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.738e-06 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659240+0000", 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.96e-06 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.985 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659246+0000", 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4999999999999999e-06 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659305+0000", 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9497999999999999e-05 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659312+0000", 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1899999999999998e-06 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.986 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664443+0000", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0051305480000000004 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664451+0000", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.5250000000000005e-06 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.666942+0000", 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002490356 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667080+0000", 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000138248 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667261+0000", 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00018106099999999999 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.988 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17359 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 311296~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.23359695400001, 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.008549785, 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17359 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659261+0000", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659262+0000", 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3799999999999998e-07 2024-09-17T21:50:05.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659264+0000", 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.582e-06 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659270+0000", 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8780000000000003e-06 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.991 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659274+0000", 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4780000000000002e-06 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659339+0000", 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5026999999999999e-05 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.659345+0000", 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4399999999999996e-06 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.992 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664779+0000", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.005433907 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.664786+0000", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1659999999999997e-06 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667645+0000", 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002859299 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667734+0000", 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9128000000000005e-05 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667811+0000", 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6441999999999998e-05 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.994 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17391 3.b 3:d5dc94fe:::rbd_data.11121f62a841.0000000000000012:head [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.22650689, 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001728929, 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17391 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.995 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666351+0000", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666352+0000", 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8699999999999995e-07 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666355+0000", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4249999999999998e-06 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666365+0000", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0043e-05 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666371+0000", 2024-09-17T21:50:05.997 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8490000000000002e-06 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666417+0000", 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6051999999999999e-05 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666426+0000", 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4940000000000003e-06 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:05.998 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667849+0000", 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001423121 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.667964+0000", 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011530400000000001 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668080+0000", 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000116054 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:05.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5516.0:17392 3.b 3:d77bf823:::rbd_data.11121f62a841.0000000000000013:head [read 319488~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 122.22647320999999, 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019355599999999, 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5516", 2024-09-17T21:50:06.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 17392 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.001 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666385+0000", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9599999999999999e-07 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666388+0000", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5739999999999998e-06 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666395+0000", 2024-09-17T21:50:06.002 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3950000000000003e-06 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666401+0000", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8259999999999998e-06 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666451+0000", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9821000000000003e-05 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:03.666458+0000", 2024-09-17T21:50:06.003 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7830000000000001e-06 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668116+0000", 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0016579000000001 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668194+0000", 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8360000000000002e-05 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.004 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:04.668320+0000", 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00012640500000000001 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23792 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.138004280000004, 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.034870534, 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.005 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23792 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.006 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754854+0000", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754855+0000", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5060000000000001e-06 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754863+0000", 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.345e-06 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754878+0000", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5311999999999998e-05 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754890+0000", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2310000000000001e-05 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754987+0000", 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7266e-05 2024-09-17T21:50:06.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754997+0000", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.5310000000000007e-06 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759460+0000", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0044626259999999999 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759471+0000", 2024-09-17T21:50:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1258e-05 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761416+0000", 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0019454459999999 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761428+0000", 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1279e-05 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.010 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787014+0000", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.025586841 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.787280+0000", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00026583599999999998 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789724+0000", 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0024439779999999999 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:273 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.137911260999999, 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0360132320000002, 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:50:06.012 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 273 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.013 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754947+0000", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4899999999999995e-07 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754952+0000", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.4150000000000001e-06 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754961+0000", 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7779999999999996e-06 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.754967+0000", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.511e-06 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755064+0000", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6766000000000002e-05 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.755070+0000", 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0009999999999998e-06 2024-09-17T21:50:06.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759611+0000", 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0045409250000000003 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759620+0000", 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.1989999999999995e-06 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.016 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761577+0000", 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.001956581 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761591+0000", 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.4382000000000001e-05 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788957+0000", 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0273655420000001 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.017 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789075+0000", 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011836 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790960+0000", 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0018851230000000001 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:274 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:50:06.018 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 96.135281313999997, 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.0335736519999998, 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 274 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757577+0000", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757579+0000", 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9089999999999998e-06 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757585+0000", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4180000000000002e-06 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757597+0000", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1844e-05 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757605+0000", 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4579999999999993e-06 2024-09-17T21:50:06.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757663+0000", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7528000000000002e-05 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting_for_map not empty", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.757669+0000", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9200000000000001e-06 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759645+0000", 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0019765749999999999 2024-09-17T21:50:06.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:29.759653+0000", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.6350000000000006e-06 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761845+0000", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.002192339 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761855+0000", 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0145e-05 2024-09-17T21:50:06.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789760+0000", 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0279042599999999 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.789824+0000", 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4286000000000001e-05 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:50:06.024 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790115+0000", 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00029083700000000001 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790977+0000", 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00086168599999999996 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791010+0000", 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3680999999999998e-05 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791069+0000", 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8746000000000003e-05 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791150+0000", 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.1384999999999994e-05 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.026 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23793 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch ping cookie 140714805737264 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e148)", 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.791738611, 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.6872306319999999, 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23793 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.027 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101119+0000", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101120+0000", 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6599999999999994e-07 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101127+0000", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3810000000000004e-06 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101136+0000", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6579999999999997e-06 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101148+0000", 2024-09-17T21:50:06.029 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1581e-05 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101222+0000", 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.3622999999999996e-05 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.101232+0000", 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0026e-05 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.030 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761460+0000", 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.660228013 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761470+0000", 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0397999999999999e-05 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788003+0000", 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0265330530000001 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.031 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788115+0000", 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000112013 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788350+0000", 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00023492000000000001 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.032 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4370.0:23794 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 140714805737264 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.136419114000006, 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0342747240000001, 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4370", 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 23794 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756439+0000", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756441+0000", 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:50:06.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756452+0000", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.0441e-05 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756477+0000", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.5680000000000001e-05 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756491+0000", 2024-09-17T21:50:06.035 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.3791000000000001e-05 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756584+0000", 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3206000000000002e-05 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op must wait for map", 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.756594+0000", 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6090000000000007e-06 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.036 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761504+0000", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0049104359999999998 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.761514+0000", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6420000000000002e-06 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788393+0000", 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.026879168 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.037 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.788458+0000", 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4777999999999995e-05 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790714+0000", 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0022557660000000002 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.6223.0:275 3.4 3:2310fb0b:::rbd_header.111257056fdb:head [watch reconnect cookie 139927551597216 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-17T21:50:06.038 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 95.130510083999994, 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.030721531, 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.6223", 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2571613959", 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 275 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.039 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762348+0000", 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762350+0000", 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5549999999999999e-06 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762355+0000", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7300000000000002e-06 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762367+0000", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1824000000000001e-05 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762377+0000", 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.6690000000000001e-06 2024-09-17T21:50:06.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762622+0000", 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00024496899999999999 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for peered", 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:30.762634+0000", 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2466999999999999e-05 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.042 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790440+0000", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.027805826 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for rw locks", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.790559+0000", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00011914800000000001 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791186+0000", 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00062735299999999998 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.791297+0000", 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000110209 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:48:31.793069+0000", 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.001772781 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.045 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:06.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:50:06.046 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:06.043+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:06.146 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:06.144+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:06.195 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:06.195 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:06.195 INFO:teuthology.orchestra.run.smithi007.stdout: "num_ops": 0 2024-09-17T21:50:06.195 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:06.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:50:06.248 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:06.245+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:06.348 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:06.346+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:06.352 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [], 2024-09-17T21:50:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "complaint_time": 30, 2024-09-17T21:50:06.352 INFO:teuthology.orchestra.run.smithi007.stdout: "num_blocked_ops": 0 2024-09-17T21:50:06.352 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:06.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:50:06.449 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:06.447+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:06.469 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:06.469 INFO:teuthology.orchestra.run.smithi007.stdout: "size": 20, 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 600, 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "ops": [ 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.4673.0:2272 3.1e 3:7f92fedf:::rbd_data.1112abb63c65.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 211.59368786300001, 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.025836163999999998, 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:06.470 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.4673", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2788608600", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2272 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872773+0000", 2024-09-17T21:50:06.471 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.872774+0000", 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.5650000000000001e-06 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880226+0000", 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0074517519999999999 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.472 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880250+0000", 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.374e-05 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880263+0000", 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.2978e-05 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880335+0000", 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2688999999999995e-05 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.880457+0000", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000122192 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "waiting for subops from 3", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.881014+0000", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00055625499999999997 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "op_commit", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.892010+0000", 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.010996752 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898540+0000", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0065297389999999997 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "commit_sent", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898578+0000", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.7608999999999999e-05 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:46:34.898609+0000", 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.0892999999999997e-05 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2573 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 753664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55724155300001, 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022645460999999999, 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.476 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2573 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909219+0000", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.39e-07 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909221+0000", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.897e-06 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909229+0000", 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.8930000000000005e-06 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909234+0000", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9030000000000003e-06 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931753+0000", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022519517999999999 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931794+0000", 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0241000000000003e-05 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931864+0000", 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0669999999999999e-05 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2574 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 741376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55722025200001, 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022922557999999999, 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.480 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2574 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909240+0000", 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909241+0000", 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.4900000000000001e-07 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.482 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909243+0000", 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2220000000000001e-06 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909249+0000", 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.4450000000000001e-06 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909254+0000", 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.8289999999999998e-06 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.483 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.931910+0000", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022656081000000002 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932000+0000", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.9689000000000001e-05 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932163+0000", 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000162943 2024-09-17T21:50:06.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2575 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 763904~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55719949100001, 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.0230402, 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.485 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2575 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.486 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909261+0000", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.77e-07 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909263+0000", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.1019999999999999e-06 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909270+0000", 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5169999999999996e-06 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.487 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909275+0000", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.882e-06 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932179+0000", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022904627 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932218+0000", 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8726000000000002e-05 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.488 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932301+0000", 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2968999999999998e-05 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2576 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 751616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55717838499999, 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023193844000000002, 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.489 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2576 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:06.490 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909282+0000", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.03e-07 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909285+0000", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.198e-06 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909291+0000", 2024-09-17T21:50:06.491 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.3799999999999999e-06 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909296+0000", 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.9849999999999997e-06 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932339+0000", 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023043251000000001 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.492 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932396+0000", 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.6458000000000001e-05 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932476+0000", 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.0168999999999998e-05 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2577 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 774144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.493 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55715789600001, 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023305287000000001, 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2577 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.494 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909303+0000", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8700000000000001e-07 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.495 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909305+0000", 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909312+0000", 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5270000000000004e-06 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909317+0000", 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.0719999999999999e-06 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.496 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932495+0000", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023178153999999999 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932538+0000", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2731999999999997e-05 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932608+0000", 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0139e-05 2024-09-17T21:50:06.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2578 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 761856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55713379900001, 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023583544000000001, 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.498 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2578 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909327+0000", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.75e-07 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909329+0000", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.2759999999999999e-06 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909336+0000", 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.6749999999999996e-06 2024-09-17T21:50:06.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909341+0000", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.3340000000000001e-06 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932653+0000", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023311449000000001 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932751+0000", 2024-09-17T21:50:06.501 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.8426999999999998e-05 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932910+0000", 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000158808 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2579 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 772096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.557111306, 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023707547999999998, 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.502 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2579 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909349+0000", 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909350+0000", 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.5000000000000002e-07 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909357+0000", 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.0509999999999999e-06 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.504 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909364+0000", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9160000000000004e-06 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909369+0000", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4330000000000003e-06 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932943+0000", 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023574178000000001 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.932985+0000", 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.1415999999999998e-05 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933057+0000", 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.1903999999999995e-05 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2580 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 784384~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.506 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.557083699, 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023954138999999999, 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2580 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.507 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909377+0000", 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3099999999999998e-07 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909379+0000", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.323e-06 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909387+0000", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.2069999999999998e-06 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909394+0000", 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.4730000000000003e-06 2024-09-17T21:50:06.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933095+0000", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023700603000000001 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933178+0000", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2989000000000001e-05 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933331+0000", 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000153113 2024-09-17T21:50:06.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:2581 3.1e 3:7db8dc2f:::rbd_data.11126af432c4.0000000000000000:25 [read 782336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111)", 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 139.55705885899999, 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.024070141999999999, 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.511 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 2581 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.512 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909402+0000", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909404+0000", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3879999999999998e-06 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909411+0000", 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.024e-06 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.513 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.909417+0000", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7409999999999998e-06 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933361+0000", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023943994 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933402+0000", 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.0926000000000001e-05 2024-09-17T21:50:06.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:46.933472+0000", 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.9666999999999996e-05 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21382 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.40510508400001, 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022730752, 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21382 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.516 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061355+0000", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061356+0000", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.1200000000000003e-07 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061359+0000", 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.926e-06 2024-09-17T21:50:06.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061366+0000", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.8029999999999999e-06 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061371+0000", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.507e-06 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083900+0000", 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022528889999999999 2024-09-17T21:50:06.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.083984+0000", 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.4254999999999999e-05 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084086+0000", 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101859 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.519 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21383 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 919552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.405081322, 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022832969000000002, 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21383 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.520 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061379+0000", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061380+0000", 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.5499999999999998e-07 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061382+0000", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.3970000000000001e-06 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061397+0000", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.521e-05 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061408+0000", 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1199000000000001e-05 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084101+0000", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02269272 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084154+0000", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.2506999999999997e-05 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084212+0000", 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.8381000000000002e-05 2024-09-17T21:50:06.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21384 3.7 3:e787327d:::rbd_data.11126af432c4.000000000000000c:25 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.40503893299999, 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023013486, 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.524 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21384 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061421+0000", 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061422+0000", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4099999999999999e-07 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061432+0000", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.7459999999999995e-06 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061443+0000", 2024-09-17T21:50:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.1375e-05 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.061450+0000", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.7649999999999996e-06 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084250+0000", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022799624000000001 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084332+0000", 2024-09-17T21:50:06.527 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2545000000000003e-05 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.084435+0000", 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.00010289 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21725 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 976896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.34929027999999, 2024-09-17T21:50:06.528 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022599402000000001, 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21725 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:06.529 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.530 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.530 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117170+0000", 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117171+0000", 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2e-07 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.532 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117173+0000", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.4399999999999999e-06 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117180+0000", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 7.199e-06 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117186+0000", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4040000000000003e-06 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.533 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139584+0000", 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022398487000000002 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139666+0000", 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.2340000000000001e-05 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139770+0000", 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000103112 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21726 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 987136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.349267185, 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022813918999999998, 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21726 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.535 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117193+0000", 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.536 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117194+0000", 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.3700000000000001e-07 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117196+0000", 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.491e-06 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117206+0000", 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.4110000000000002e-06 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.537 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117212+0000", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.1299999999999998e-06 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139819+0000", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022606867999999999 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.139906+0000", 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7718000000000003e-05 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140007+0000", 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000100864 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21727 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 997376~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.34924080499999, 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023033775999999999, 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21727 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117220+0000", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6899999999999998e-07 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117223+0000", 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.6709999999999999e-06 2024-09-17T21:50:06.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117229+0000", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.7690000000000002e-06 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117232+0000", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.3440000000000001e-06 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140047+0000", 2024-09-17T21:50:06.542 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.022815417000000001 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140134+0000", 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 8.7015000000000006e-05 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140253+0000", 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000119091 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21728 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1007616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.349224169, 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023257465000000001, 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21728 2024-09-17T21:50:06.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117236+0000", 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.545 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117237+0000", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8000000000000002e-07 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117238+0000", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.449e-06 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117241+0000", 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.2109999999999998e-06 2024-09-17T21:50:06.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117244+0000", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.8389999999999998e-06 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140300+0000", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023055777 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140392+0000", 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.2417000000000006e-05 2024-09-17T21:50:06.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140494+0000", 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.000101492 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21729 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1017856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.349194205, 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.02336361, 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21729 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.549 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117266+0000", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117267+0000", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.58e-07 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117269+0000", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.9999999999999999e-06 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.550 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117272+0000", 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.8709999999999999e-06 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117275+0000", 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.016e-06 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140527+0000", 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023251964 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.551 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140570+0000", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.2175999999999997e-05 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140630+0000", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 6.0225e-05 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21730 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1028096~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:06.552 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.34918128000001, 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023485903999999998, 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21730 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:06.553 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117279+0000", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.05e-07 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117281+0000", 2024-09-17T21:50:06.554 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.268e-06 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117284+0000", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.976e-06 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117286+0000", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 2.7599999999999998e-06 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140663+0000", 2024-09-17T21:50:06.555 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023376793 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140711+0000", 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.7475999999999997e-05 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140765+0000", 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4425999999999997e-05 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "description": "osd_op(client.5808.0:21731 3.18 3:1dada8b5:::rbd_data.11126af432c4.000000000000000d:25 [read 1038336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114)", 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "initiated_at": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "age": 135.34917105900001, 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023650035, 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "continuous": false, 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type_data": { 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "flag_point": "started", 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_info": { 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client": "client.5808", 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "client_addr": "172.21.15.165:0/2045567827", 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "tid": 21731 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.557 INFO:teuthology.orchestra.run.smithi007.stdout: "events": [ 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "initiated", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "header_read", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117289+0000", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "throttled", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117290+0000", 2024-09-17T21:50:06.558 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 3.27e-07 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "all_read", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117291+0000", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 1.7209999999999999e-06 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "dispatched", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117297+0000", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.4349999999999996e-06 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "queued_for_pg", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.117301+0000", 2024-09-17T21:50:06.559 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 4.6199999999999998e-06 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "reached_pg", 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140786+0000", 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 0.023484824000000001 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "started", 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140846+0000", 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 5.9505e-05 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "event": "done", 2024-09-17T21:50:06.560 INFO:teuthology.orchestra.run.smithi007.stdout: "time": "2024-09-17T21:47:51.140939+0000", 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: "duration": 9.3603000000000005e-05 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-17T21:50:06.561 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:06.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:50:06.563 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:06.560+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:06.663 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:06.661+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:06.681 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:06.682 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:06.682 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:06.682 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:06.691 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:50:06.776 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:06.774+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:50:06.799 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:50:06.818 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:50:06.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:06.846 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:06.846 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:06.846 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:06.855 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:50:06.897 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:06.895+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:06.966 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:06.964+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.008 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3597 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 0~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 221.26525724499999, 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99018109899999995, 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3597 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738969+0000", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738971+0000", 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.68e-06 2024-09-17T21:50:07.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738978+0000", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8889999999999996e-06 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.738999+0000", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0852e-05 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739008+0000", 2024-09-17T21:50:07.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6980000000000004e-06 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739083+0000", 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5840999999999994e-05 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.739098+0000", 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4482e-05 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.728834+0000", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98973593000000004 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729027+0000", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019330500000000001 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729151+0000", 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012342200000000001 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4672.0:3598 3.1 3:87faf68b:::rbd_data.11121f8e87bb.0000000000000048:head [read 4096~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 221.26421278500001, 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98947411299999999, 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4672", 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1549460246", 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3598 2024-09-17T21:50:07.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740014+0000", 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740015+0000", 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0300000000000001e-06 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740023+0000", 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1380000000000007e-06 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740035+0000", 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1702000000000001e-05 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740042+0000", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.25e-06 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740107+0000", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5017000000000004e-05 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:25.740113+0000", 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4359999999999999e-06 2024-09-17T21:50:07.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729338+0000", 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.98922413899999995 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729392+0000", 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4450999999999998e-05 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:26.729488+0000", 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5950000000000001e-05 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4370.0:14494 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e97)", 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 160.06302745400001, 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.004956438, 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4370", 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2702324635", 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14494 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941199+0000", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941201+0000", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.012e-06 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941210+0000", 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4489999999999999e-06 2024-09-17T21:50:07.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941227+0000", 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6988999999999999e-05 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941238+0000", 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1744e-05 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941321+0000", 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2164999999999997e-05 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.941326+0000", 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.147e-06 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943459+0000", 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021328290000000001 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:26.943468+0000", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6129999999999993e-06 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945140+0000", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.00167194 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.945322+0000", 2024-09-17T21:50:07.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018189200000000001 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946037+0000", 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000714888 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946088+0000", 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1008999999999999e-05 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:27.946156+0000", 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7761000000000004e-05 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3.e27131e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.847109786, 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.89212046899999997, 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157117+0000", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157119+0000", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7129999999999999e-06 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157127+0000", 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6179999999999995e-06 2024-09-17T21:50:07.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157146+0000", 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8669e-05 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157155+0000", 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1800000000000002e-06 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157232+0000", 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6490999999999997e-05 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.157244+0000", 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2883e-05 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049222+0000", 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.891977935 2024-09-17T21:50:07.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049237+0000", 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4980000000000001e-05 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:13509 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.644673044, 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69210735999999995, 2024-09-17T21:50:07.031 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 13509 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359554+0000", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359555+0000", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.683e-06 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359562+0000", 2024-09-17T21:50:07.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7290000000000003e-06 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359573+0000", 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0882e-05 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359584+0000", 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0730000000000001e-05 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359651+0000", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7600000000000003e-05 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359660+0000", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1540000000000008e-06 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049320+0000", 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68965955099999998 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049453+0000", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013259400000000001 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049867+0000", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041440900000000002 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051524+0000", 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00165686 2024-09-17T21:50:07.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051569+0000", 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5201000000000003e-05 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051625+0000", 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5760999999999997e-05 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051661+0000", 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6205999999999999e-05 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14685 3.9 3:978c8e47:::rbd_object_map.11125d2318a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.644615864, 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.692413897, 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14685 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359611+0000", 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7899999999999999e-07 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359614+0000", 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5210000000000001e-06 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359625+0000", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0976e-05 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359631+0000", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7259999999999996e-06 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359695+0000", 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3742000000000001e-05 2024-09-17T21:50:07.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.359715+0000", 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9426e-05 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050488+0000", 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69077383599999997 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050568+0000", 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9532999999999999e-05 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050975+0000", 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040697299999999998 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051679+0000", 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070374200000000004 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051921+0000", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024247099999999999 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051982+0000", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1297999999999995e-05 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052025+0000", 2024-09-17T21:50:07.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2274000000000002e-05 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14826 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 0~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.644045283, 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68926441100000002, 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14826 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360181+0000", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360182+0000", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360184+0000", 2024-09-17T21:50:07.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3130000000000001e-06 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360190+0000", 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1140000000000002e-06 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360195+0000", 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.113e-06 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360245+0000", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0092000000000001e-05 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360254+0000", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.884e-06 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.048924+0000", 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68866959999999999 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049066+0000", 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014255999999999999 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049446+0000", 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000379331 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.050 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14827 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64401734899999, 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68991999800000003, 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14827 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360209+0000", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360210+0000", 2024-09-17T21:50:07.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1800000000000001e-07 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360212+0000", 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360220+0000", 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1159999999999999e-06 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360226+0000", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.665e-06 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360285+0000", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919000000000003e-05 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360295+0000", 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7410000000000004e-06 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049852+0000", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68955738099999997 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.049897+0000", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5124999999999999e-05 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050129+0000", 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023216400000000001 2024-09-17T21:50:07.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14889 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 8192~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64388299699999, 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69013807500000002, 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14889 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360344+0000", 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4200000000000002e-07 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360346+0000", 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9369999999999998e-06 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360351+0000", 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4840000000000003e-06 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360356+0000", 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.425e-06 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360398+0000", 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1671000000000001e-05 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360405+0000", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5580000000000003e-06 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050159+0000", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68975378600000004 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050266+0000", 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106699 2024-09-17T21:50:07.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050482+0000", 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021617300000000001 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14890 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 32768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64386146199999, 2024-09-17T21:50:07.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69061399000000001, 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14890 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:07.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360365+0000", 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360366+0000", 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.15e-07 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360368+0000", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2019999999999998e-06 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360374+0000", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1460000000000001e-06 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360380+0000", 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4330000000000001e-06 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360432+0000", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2082e-05 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360440+0000", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9179999999999994e-06 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050511+0000", 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007081800000003 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050579+0000", 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8106000000000001e-05 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.050979+0000", 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039986999999999998 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14958 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 40960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.643700497, 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69080024799999995, 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14958 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360526+0000", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360527+0000", 2024-09-17T21:50:07.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7099999999999997e-07 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360529+0000", 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.469e-06 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360535+0000", 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0970000000000002e-06 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360540+0000", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0839999999999999e-06 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360579+0000", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8847000000000003e-05 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360586+0000", 2024-09-17T21:50:07.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8190000000000003e-06 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051009+0000", 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69042360000000003 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051082+0000", 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2585999999999999e-05 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051326+0000", 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00024437500000000002 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:14960 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 65536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64364473200001, 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69111932300000001, 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14960 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360582+0000", 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8599999999999999e-07 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360585+0000", 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3800000000000001e-06 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360590+0000", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5430000000000001e-06 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360595+0000", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1019999999999996e-06 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360659+0000", 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3497999999999994e-05 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.360666+0000", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3710000000000002e-06 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051357+0000", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.690690782 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051443+0000", 2024-09-17T21:50:07.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5921000000000006e-05 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051701+0000", 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025834000000000002 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15023 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 49152~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.642298125, 2024-09-17T21:50:07.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69007157799999996, 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15023 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361929+0000", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9299999999999999e-07 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361930+0000", 2024-09-17T21:50:07.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.59e-06 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361935+0000", 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8239999999999999e-06 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361938+0000", 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6589999999999999e-06 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361995+0000", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7445999999999999e-05 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362006+0000", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0190000000000001e-05 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051722+0000", 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68971642200000005 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.051774+0000", 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2308000000000001e-05 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052000+0000", 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022584600000000001 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15025 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 73728~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64224656900001, 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69031700399999996, 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15025 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361980+0000", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361981+0000", 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000002e-07 2024-09-17T21:50:07.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361983+0000", 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8009999999999999e-06 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361989+0000", 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0800000000000002e-06 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.361994+0000", 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3239999999999996e-06 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362031+0000", 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7320999999999997e-05 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362039+0000", 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.469e-06 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052041+0000", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69000256400000004 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052128+0000", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6786000000000006e-05 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052297+0000", 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016925499999999999 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.087 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15090 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 81920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64155830799999, 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689882773, 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.088 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15090 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362668+0000", 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362669+0000", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.16e-07 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362673+0000", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5499999999999996e-06 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362677+0000", 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8850000000000001e-06 2024-09-17T21:50:07.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362680+0000", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7159999999999999e-06 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362724+0000", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4382000000000003e-05 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362732+0000", 2024-09-17T21:50:07.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8620000000000003e-06 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052332+0000", 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68959984200000002 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052383+0000", 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1566000000000001e-05 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052551+0000", 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016775399999999999 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15092 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 98304~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.641541257, 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.69009827199999996, 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15092 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362685+0000", 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362686+0000", 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.27e-07 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362688+0000", 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9810000000000002e-06 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362694+0000", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8370000000000002e-06 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362698+0000", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9430000000000002e-06 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362753+0000", 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5631999999999999e-05 2024-09-17T21:50:07.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.362758+0000", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3460000000000001e-06 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052581+0000", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68982222199999998 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052635+0000", 2024-09-17T21:50:07.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4372000000000001e-05 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052784+0000", 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014861200000000001 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15161 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 90112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64060301699999, 2024-09-17T21:50:07.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689381401, 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15161 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363624+0000", 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.77e-07 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363626+0000", 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3870000000000002e-06 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363634+0000", 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5270000000000001e-06 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363641+0000", 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6479999999999997e-06 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363687+0000", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6267999999999998e-05 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363695+0000", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2630000000000003e-06 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052813+0000", 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68911825999999998 2024-09-17T21:50:07.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.052860+0000", 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6986e-05 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053005+0000", 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000144685 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5417.0:15162 3.d 3:b1951a30:::rbd_data.11121f62a841.0000000000000011:27 [read 114688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e120)", 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 128.64057267699999, 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.689549474, 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5417", 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2158814402", 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15162 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363654+0000", 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5900000000000002e-07 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363657+0000", 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4820000000000001e-06 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363665+0000", 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6899999999999992e-06 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363671+0000", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8279999999999997e-06 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363722+0000", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0971999999999999e-05 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:58.363730+0000", 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4479999999999997e-06 2024-09-17T21:50:07.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053032+0000", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.68930248900000002 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053074+0000", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1143000000000002e-05 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:47:59.053203+0000", 2024-09-17T21:50:07.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129963 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17352 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch reconnect cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 123.346175229, 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0127286529999999, 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17352 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658051+0000", 2024-09-17T21:50:07.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658053+0000", 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.361e-06 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658059+0000", 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1290000000000001e-06 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658071+0000", 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658079+0000", 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.311e-06 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658153+0000", 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4790999999999996e-05 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.658161+0000", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2790000000000001e-06 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659865+0000", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017041440000000001 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:03.659874+0000", 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1109999999999997e-06 2024-09-17T21:50:07.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668220+0000", 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0083462729999999 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.668479+0000", 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025881400000000001 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670780+0000", 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023010610000000001 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5516.0:17419 3.1 3:821d1c11:::rbd_header.11121f62a841:head [watch ping cookie 140353759886912 gen 13] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 122.445057529, 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.11104083300000001, 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5516", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/1545881044", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17419 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559169+0000", 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559170+0000", 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9699999999999994e-07 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559177+0000", 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7100000000000001e-06 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559190+0000", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3416e-05 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559203+0000", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2993000000000001e-05 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559309+0000", 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106119 2024-09-17T21:50:07.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.559318+0000", 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5299999999999996e-06 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669864+0000", 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.110546056 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.669967+0000", 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102673 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:04.670210+0000", 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000243339 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.120 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:07.122 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:50:07.122 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:07.120+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:07.155 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:07.156 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:07.167 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-17T21:50:06.798000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308585257,"num_objects":885,"num_object_clones":24,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":723,"num_read":380754,"num_read_kb":1875763,"num_write":22749,"num_write_kb":604706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21082,"ondisk_log_size":21082,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":517316,"kb_used_data":158424,"kb_used_omap":36,"kb_used_meta":356891,"kb_avail":565713724,"statfs":{"total":579820584960,"available":579290853376,"internally_reserved":0,"allocated":162226176,"data_stored":239317618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":365456681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-251303523,"num_objects":-264,"num_object_clones":-80,"num_object_copies":-528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-264,"num_whiteouts":-50,"num_read":1320,"num_read_kb":1121,"num_write":273,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015451"},"pg_stats":[{"pgid":"3.1f","version":"241'404","reported_seq":11842,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.768819+0000","last_change":"2024-09-17T21:50:03.693789+0000","last_active":"2024-09-17T21:50:03.768819+0000","last_peered":"2024-09-17T21:50:03.768819+0000","last_clean":"2024-09-17T21:50:03.768819+0000","last_became_active":"2024-09-17T21:49:56.760149+0000","last_became_peered":"2024-09-17T21:49:56.760149+0000","last_unstale":"2024-09-17T21:50:03.768819+0000","last_undegraded":"2024-09-17T21:50:03.768819+0000","last_fullsized":"2024-09-17T21:50:03.768819+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:39.699036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011878212000000001,"stat_sum":{"num_bytes":17712693,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":11470,"num_read_kb":71057,"num_write":367,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.1e","version":"243'601","reported_seq":20623,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.900065+0000","last_change":"2024-09-17T21:50:05.806530+0000","last_active":"2024-09-17T21:50:05.900065+0000","last_peered":"2024-09-17T21:50:05.900065+0000","last_clean":"2024-09-17T21:50:05.900065+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:05.900065+0000","last_undegraded":"2024-09-17T21:50:05.900065+0000","last_fullsized":"2024-09-17T21:50:05.900065+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:26.800446+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014483848000000001,"stat_sum":{"num_bytes":17454592,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":26,"num_read":19155,"num_read_kb":109404,"num_write":453,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1d","version":"242'415","reported_seq":5517,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.794366+0000","last_change":"2024-09-17T21:50:05.795063+0000","last_active":"2024-09-17T21:50:05.794366+0000","last_peered":"2024-09-17T21:50:05.794366+0000","last_clean":"2024-09-17T21:50:05.794366+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:05.794366+0000","last_undegraded":"2024-09-17T21:50:05.794366+0000","last_fullsized":"2024-09-17T21:50:05.794366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:38.708630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318100000000003,"stat_sum":{"num_bytes":1148416,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":665,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1c","version":"243'656","reported_seq":18601,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.945876+0000","last_change":"2024-09-17T21:50:05.805124+0000","last_active":"2024-09-17T21:50:05.945876+0000","last_peered":"2024-09-17T21:50:05.945876+0000","last_clean":"2024-09-17T21:50:05.945876+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:05.945876+0000","last_undegraded":"2024-09-17T21:50:05.945876+0000","last_fullsized":"2024-09-17T21:50:05.945876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:26.711241+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010758909000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19883,"num_read_kb":87050,"num_write":713,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1b","version":"242'461","reported_seq":16978,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.977258+0000","last_change":"2024-09-17T21:50:04.797063+0000","last_active":"2024-09-17T21:50:04.977258+0000","last_peered":"2024-09-17T21:50:04.977258+0000","last_clean":"2024-09-17T21:50:04.977258+0000","last_became_active":"2024-09-17T21:49:56.760843+0000","last_became_peered":"2024-09-17T21:49:56.760843+0000","last_unstale":"2024-09-17T21:50:04.977258+0000","last_undegraded":"2024-09-17T21:50:04.977258+0000","last_fullsized":"2024-09-17T21:50:04.977258+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:46:29.467956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062440200000000003,"stat_sum":{"num_bytes":36542976,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":17,"num_read":14439,"num_read_kb":43518,"num_write":364,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.1a","version":"242'727","reported_seq":12121,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.980670+0000","last_change":"2024-09-17T21:50:04.797438+0000","last_active":"2024-09-17T21:50:04.980670+0000","last_peered":"2024-09-17T21:50:04.980670+0000","last_clean":"2024-09-17T21:50:04.980670+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:04.980670+0000","last_undegraded":"2024-09-17T21:50:04.980670+0000","last_fullsized":"2024-09-17T21:50:04.980670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:30.039284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097771100000000003,"stat_sum":{"num_bytes":13631594,"num_objects":28,"num_object_clones":1,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":24,"num_read":14100,"num_read_kb":82620,"num_write":737,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.19","version":"241'464","reported_seq":5188,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.723435+0000","last_change":"2024-09-17T21:50:03.724020+0000","last_active":"2024-09-17T21:50:03.723435+0000","last_peered":"2024-09-17T21:50:03.723435+0000","last_clean":"2024-09-17T21:50:03.723435+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:03.723435+0000","last_undegraded":"2024-09-17T21:50:03.723435+0000","last_fullsized":"2024-09-17T21:50:03.723435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:21.625796+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029561269999999998,"stat_sum":{"num_bytes":1949184,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":456,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.18","version":"242'708","reported_seq":13319,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.942423+0000","last_change":"2024-09-17T21:50:05.793650+0000","last_active":"2024-09-17T21:50:05.942423+0000","last_peered":"2024-09-17T21:50:05.942423+0000","last_clean":"2024-09-17T21:50:05.942423+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:05.942423+0000","last_undegraded":"2024-09-17T21:50:05.942423+0000","last_fullsized":"2024-09-17T21:50:05.942423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:26:37.877005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001869157,"stat_sum":{"num_bytes":24559616,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":22,"num_read":12422,"num_read_kb":69065,"num_write":806,"num_write_kb":13809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.17","version":"242'385","reported_seq":7823,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.800256+0000","last_change":"2024-09-17T21:50:05.801235+0000","last_active":"2024-09-17T21:50:05.800256+0000","last_peered":"2024-09-17T21:50:05.800256+0000","last_clean":"2024-09-17T21:50:05.800256+0000","last_became_active":"2024-09-17T21:49:56.761187+0000","last_became_peered":"2024-09-17T21:49:56.761187+0000","last_unstale":"2024-09-17T21:50:05.800256+0000","last_undegraded":"2024-09-17T21:50:05.800256+0000","last_fullsized":"2024-09-17T21:50:05.800256+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:19.656714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066466060000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.16","version":"242'1024","reported_seq":12045,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.807436+0000","last_change":"2024-09-17T21:50:05.808458+0000","last_active":"2024-09-17T21:50:05.807436+0000","last_peered":"2024-09-17T21:50:05.807436+0000","last_clean":"2024-09-17T21:50:05.807436+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:05.807436+0000","last_undegraded":"2024-09-17T21:50:05.807436+0000","last_fullsized":"2024-09-17T21:50:05.807436+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:36:09.441372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01224691,"stat_sum":{"num_bytes":770048,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1044,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.15","version":"240'783","reported_seq":10226,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.693905+0000","last_change":"2024-09-17T21:50:03.695078+0000","last_active":"2024-09-17T21:50:03.693905+0000","last_peered":"2024-09-17T21:50:03.693905+0000","last_clean":"2024-09-17T21:50:03.693905+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:03.693905+0000","last_undegraded":"2024-09-17T21:50:03.693905+0000","last_fullsized":"2024-09-17T21:50:03.693905+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:28.221184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013107594,"stat_sum":{"num_bytes":2616832,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":729,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.14","version":"240'518","reported_seq":19353,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681132+0000","last_change":"2024-09-17T21:50:03.683218+0000","last_active":"2024-09-17T21:50:03.681132+0000","last_peered":"2024-09-17T21:50:03.681132+0000","last_clean":"2024-09-17T21:50:03.681132+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:03.681132+0000","last_undegraded":"2024-09-17T21:50:03.681132+0000","last_fullsized":"2024-09-17T21:50:03.681132+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:08.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028355059999999998,"stat_sum":{"num_bytes":17027088,"num_objects":30,"num_object_clones":1,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":18,"num_read":18243,"num_read_kb":143651,"num_write":560,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.682083+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:03.682083+0000","last_peered":"2024-09-17T21:50:03.682083+0000","last_clean":"2024-09-17T21:50:03.682083+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:03.682083+0000","last_undegraded":"2024-09-17T21:50:03.682083+0000","last_fullsized":"2024-09-17T21:50:03.682083+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"241'749","reported_seq":10817,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.778889+0000","last_change":"2024-09-17T21:50:03.725924+0000","last_active":"2024-09-17T21:50:03.778889+0000","last_peered":"2024-09-17T21:50:03.778889+0000","last_clean":"2024-09-17T21:50:03.778889+0000","last_became_active":"2024-09-17T21:49:56.760413+0000","last_became_peered":"2024-09-17T21:49:56.760413+0000","last_unstale":"2024-09-17T21:50:03.778889+0000","last_undegraded":"2024-09-17T21:50:03.778889+0000","last_fullsized":"2024-09-17T21:50:03.778889+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:25.690688+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0048677659999999999,"stat_sum":{"num_bytes":4534311,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":12158,"num_read_kb":53859,"num_write":609,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":491,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.720974+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:03.720974+0000","last_peered":"2024-09-17T21:50:03.720974+0000","last_clean":"2024-09-17T21:50:03.720974+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:03.720974+0000","last_undegraded":"2024-09-17T21:50:03.720974+0000","last_fullsized":"2024-09-17T21:50:03.720974+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"242'567","reported_seq":6875,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.791329+0000","last_change":"2024-09-17T21:50:05.792532+0000","last_active":"2024-09-17T21:50:05.791329+0000","last_peered":"2024-09-17T21:50:05.791329+0000","last_clean":"2024-09-17T21:50:05.791329+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:05.791329+0000","last_undegraded":"2024-09-17T21:50:05.791329+0000","last_fullsized":"2024-09-17T21:50:05.791329+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:20.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382643,"stat_sum":{"num_bytes":975872,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681599+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:03.681599+0000","last_peered":"2024-09-17T21:50:03.681599+0000","last_clean":"2024-09-17T21:50:03.681599+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:03.681599+0000","last_undegraded":"2024-09-17T21:50:03.681599+0000","last_fullsized":"2024-09-17T21:50:03.681599+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"241'702","reported_seq":19480,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.726102+0000","last_change":"2024-09-17T21:50:03.726394+0000","last_active":"2024-09-17T21:50:03.726102+0000","last_peered":"2024-09-17T21:50:03.726102+0000","last_clean":"2024-09-17T21:50:03.726102+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:03.726102+0000","last_undegraded":"2024-09-17T21:50:03.726102+0000","last_fullsized":"2024-09-17T21:50:03.726102+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:00.144007+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051008770000000002,"stat_sum":{"num_bytes":3265536,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":502,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":500,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.720975+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:03.720975+0000","last_peered":"2024-09-17T21:50:03.720975+0000","last_clean":"2024-09-17T21:50:03.720975+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:03.720975+0000","last_undegraded":"2024-09-17T21:50:03.720975+0000","last_fullsized":"2024-09-17T21:50:03.720975+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"242'547","reported_seq":13072,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.925034+0000","last_change":"2024-09-17T21:50:05.800180+0000","last_active":"2024-09-17T21:50:05.925034+0000","last_peered":"2024-09-17T21:50:05.925034+0000","last_clean":"2024-09-17T21:50:05.925034+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:05.925034+0000","last_undegraded":"2024-09-17T21:50:05.925034+0000","last_fullsized":"2024-09-17T21:50:05.925034+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:37.894138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057336660000000001,"stat_sum":{"num_bytes":16,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":530,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.3","version":"242'1028","reported_seq":11932,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.949563+0000","last_change":"2024-09-17T21:50:04.797777+0000","last_active":"2024-09-17T21:50:04.949563+0000","last_peered":"2024-09-17T21:50:04.949563+0000","last_clean":"2024-09-17T21:50:04.949563+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:04.949563+0000","last_undegraded":"2024-09-17T21:50:04.949563+0000","last_fullsized":"2024-09-17T21:50:04.949563+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:52.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301394,"stat_sum":{"num_bytes":2097152,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":16,"num_read":14292,"num_read_kb":38832,"num_write":1258,"num_write_kb":10670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"2.2","version":"19'2","reported_seq":471,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.794272+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:05.794272+0000","last_peered":"2024-09-17T21:50:05.794272+0000","last_clean":"2024-09-17T21:50:05.794272+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:05.794272+0000","last_undegraded":"2024-09-17T21:50:05.794272+0000","last_fullsized":"2024-09-17T21:50:05.794272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"241'449","reported_seq":7324,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.728763+0000","last_change":"2024-09-17T21:50:03.728763+0000","last_active":"2024-09-17T21:50:03.728763+0000","last_peered":"2024-09-17T21:50:03.728763+0000","last_clean":"2024-09-17T21:50:03.728763+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:03.728763+0000","last_undegraded":"2024-09-17T21:50:03.728763+0000","last_fullsized":"2024-09-17T21:50:03.728763+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:53:21.265305+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072556410000000002,"stat_sum":{"num_bytes":11170923,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":23,"num_read":10031,"num_read_kb":30206,"num_write":352,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.790822+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:05.790822+0000","last_peered":"2024-09-17T21:50:05.790822+0000","last_clean":"2024-09-17T21:50:05.790822+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:05.790822+0000","last_undegraded":"2024-09-17T21:50:05.790822+0000","last_fullsized":"2024-09-17T21:50:05.790822+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"242'1890","reported_seq":9198,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.999260+0000","last_change":"2024-09-17T21:50:04.798245+0000","last_active":"2024-09-17T21:50:04.999260+0000","last_peered":"2024-09-17T21:50:04.999260+0000","last_clean":"2024-09-17T21:50:04.999260+0000","last_became_active":"2024-09-17T21:49:56.761242+0000","last_became_peered":"2024-09-17T21:49:56.761242+0000","last_unstale":"2024-09-17T21:50:04.999260+0000","last_undegraded":"2024-09-17T21:50:04.999260+0000","last_fullsized":"2024-09-17T21:50:04.999260+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1890,"log_dups_size":0,"ondisk_log_size":1890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:03.394784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001900236,"stat_sum":{"num_bytes":2833003,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":15736,"num_read_kb":56977,"num_write":2767,"num_write_kb":12581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.679810+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:03.679810+0000","last_peered":"2024-09-17T21:50:03.679810+0000","last_clean":"2024-09-17T21:50:03.679810+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:03.679810+0000","last_undegraded":"2024-09-17T21:50:03.679810+0000","last_fullsized":"2024-09-17T21:50:03.679810+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"241'509","reported_seq":5769,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.726576+0000","last_change":"2024-09-17T21:50:03.726576+0000","last_active":"2024-09-17T21:50:03.726576+0000","last_peered":"2024-09-17T21:50:03.726576+0000","last_clean":"2024-09-17T21:50:03.726576+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:03.726576+0000","last_undegraded":"2024-09-17T21:50:03.726576+0000","last_fullsized":"2024-09-17T21:50:03.726576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:42:33.439143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052605259999999997,"stat_sum":{"num_bytes":27212816,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":5949,"num_read_kb":32245,"num_write":561,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.3","version":"17'1","reported_seq":488,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.795361+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:05.795361+0000","last_peered":"2024-09-17T21:50:05.795361+0000","last_clean":"2024-09-17T21:50:05.795361+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:05.795361+0000","last_undegraded":"2024-09-17T21:50:05.795361+0000","last_fullsized":"2024-09-17T21:50:05.795361+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":612,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.679847+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:03.679847+0000","last_peered":"2024-09-17T21:50:03.679847+0000","last_clean":"2024-09-17T21:50:03.679847+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:03.679847+0000","last_undegraded":"2024-09-17T21:50:03.679847+0000","last_fullsized":"2024-09-17T21:50:03.679847+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'487","reported_seq":7420,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.792886+0000","last_change":"2024-09-17T21:50:05.792886+0000","last_active":"2024-09-17T21:50:05.792886+0000","last_peered":"2024-09-17T21:50:05.792886+0000","last_clean":"2024-09-17T21:50:05.792886+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:05.792886+0000","last_undegraded":"2024-09-17T21:50:05.792886+0000","last_fullsized":"2024-09-17T21:50:05.792886+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:24:41.619178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132755,"stat_sum":{"num_bytes":12390400,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":24,"num_read":7499,"num_read_kb":58807,"num_write":530,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.9","version":"242'833","reported_seq":11119,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.891307+0000","last_change":"2024-09-17T21:50:04.798107+0000","last_active":"2024-09-17T21:50:04.891307+0000","last_peered":"2024-09-17T21:50:04.891307+0000","last_clean":"2024-09-17T21:50:04.891307+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:04.891307+0000","last_undegraded":"2024-09-17T21:50:04.891307+0000","last_fullsized":"2024-09-17T21:50:04.891307+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:00.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014670569999999999,"stat_sum":{"num_bytes":22810148,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":15731,"num_read_kb":70297,"num_write":782,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.a","version":"241'537","reported_seq":5388,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.689764+0000","last_change":"2024-09-17T21:50:03.691204+0000","last_active":"2024-09-17T21:50:03.689764+0000","last_peered":"2024-09-17T21:50:03.689764+0000","last_clean":"2024-09-17T21:50:03.689764+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:03.689764+0000","last_undegraded":"2024-09-17T21:50:03.689764+0000","last_fullsized":"2024-09-17T21:50:03.689764+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:58.687796+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010337094999999999,"stat_sum":{"num_bytes":921600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":4922,"num_read_kb":14304,"num_write":560,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.b","version":"241'584","reported_seq":11022,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.731084+0000","last_change":"2024-09-17T21:50:03.731084+0000","last_active":"2024-09-17T21:50:03.731084+0000","last_peered":"2024-09-17T21:50:03.731084+0000","last_clean":"2024-09-17T21:50:03.731084+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:03.731084+0000","last_undegraded":"2024-09-17T21:50:03.731084+0000","last_fullsized":"2024-09-17T21:50:03.731084+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:52.993970+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.0097800590000000007,"stat_sum":{"num_bytes":1601536,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":537,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.c","version":"242'604","reported_seq":19339,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.798643+0000","last_change":"2024-09-17T21:50:05.799346+0000","last_active":"2024-09-17T21:50:05.798643+0000","last_peered":"2024-09-17T21:50:05.798643+0000","last_clean":"2024-09-17T21:50:05.798643+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:05.798643+0000","last_undegraded":"2024-09-17T21:50:05.798643+0000","last_fullsized":"2024-09-17T21:50:05.798643+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:21:53.665692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049379510000000003,"stat_sum":{"num_bytes":13053991,"num_objects":27,"num_object_clones":1,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":22,"num_read":18226,"num_read_kb":77054,"num_write":590,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.d","version":"241'854","reported_seq":10109,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.772007+0000","last_change":"2024-09-17T21:50:03.697687+0000","last_active":"2024-09-17T21:50:03.772007+0000","last_peered":"2024-09-17T21:50:03.772007+0000","last_clean":"2024-09-17T21:50:03.772007+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:03.772007+0000","last_undegraded":"2024-09-17T21:50:03.772007+0000","last_fullsized":"2024-09-17T21:50:03.772007+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:18.562267+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015311125,"stat_sum":{"num_bytes":14479398,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":11772,"num_read_kb":74382,"num_write":929,"num_write_kb":32181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.e","version":"242'436","reported_seq":6108,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.805201+0000","last_change":"2024-09-17T21:50:05.806629+0000","last_active":"2024-09-17T21:50:05.805201+0000","last_peered":"2024-09-17T21:50:05.805201+0000","last_clean":"2024-09-17T21:50:05.805201+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:05.805201+0000","last_undegraded":"2024-09-17T21:50:05.805201+0000","last_fullsized":"2024-09-17T21:50:05.805201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:43.025624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010640322000000001,"stat_sum":{"num_bytes":3584000,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":596,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.f","version":"243'1028","reported_seq":17702,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.810367+0000","last_change":"2024-09-17T21:50:05.810367+0000","last_active":"2024-09-17T21:50:05.810367+0000","last_peered":"2024-09-17T21:50:05.810367+0000","last_clean":"2024-09-17T21:50:05.810367+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:05.810367+0000","last_undegraded":"2024-09-17T21:50:05.810367+0000","last_fullsized":"2024-09-17T21:50:05.810367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:02.255879+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015530074,"stat_sum":{"num_bytes":442880,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1358,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.10","version":"243'621","reported_seq":7408,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.798564+0000","last_change":"2024-09-17T21:50:05.798564+0000","last_active":"2024-09-17T21:50:05.798564+0000","last_peered":"2024-09-17T21:50:05.798564+0000","last_clean":"2024-09-17T21:50:05.798564+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:05.798564+0000","last_undegraded":"2024-09-17T21:50:05.798564+0000","last_fullsized":"2024-09-17T21:50:05.798564+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:52:31.274129+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0044640950000000004,"stat_sum":{"num_bytes":1048576,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":640,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.11","version":"240'544","reported_seq":6376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681055+0000","last_change":"2024-09-17T21:50:03.682939+0000","last_active":"2024-09-17T21:50:03.681055+0000","last_peered":"2024-09-17T21:50:03.681055+0000","last_clean":"2024-09-17T21:50:03.681055+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:03.681055+0000","last_undegraded":"2024-09-17T21:50:03.681055+0000","last_fullsized":"2024-09-17T21:50:03.681055+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:38.810907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002839084,"stat_sum":{"num_bytes":18665472,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":25,"num_read":4772,"num_read_kb":61222,"num_write":581,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.12","version":"239'387","reported_seq":12689,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.724097+0000","last_change":"2024-09-17T21:50:03.724554+0000","last_active":"2024-09-17T21:50:03.724097+0000","last_peered":"2024-09-17T21:50:03.724097+0000","last_clean":"2024-09-17T21:50:03.724097+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:03.724097+0000","last_undegraded":"2024-09-17T21:50:03.724097+0000","last_fullsized":"2024-09-17T21:50:03.724097+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:53.946554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033849959999999999,"stat_sum":{"num_bytes":10022912,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":463,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.13","version":"241'460","reported_seq":10372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.692288+0000","last_change":"2024-09-17T21:50:03.693166+0000","last_active":"2024-09-17T21:50:03.692288+0000","last_peered":"2024-09-17T21:50:03.692288+0000","last_clean":"2024-09-17T21:50:03.692288+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:03.692288+0000","last_undegraded":"2024-09-17T21:50:03.692288+0000","last_fullsized":"2024-09-17T21:50:03.692288+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:33.263270+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.012620528000000001,"stat_sum":{"num_bytes":23471289,"num_objects":34,"num_object_clones":2,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":8737,"num_read_kb":42437,"num_write":558,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":307994870,"num_objects":881,"num_object_clones":24,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":723,"num_read":380638,"num_read_kb":1875663,"num_write":22588,"num_write_kb":602288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120020992,"data_stored":194568424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":20962,"ondisk_log_size":20962,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61092,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310748,"statfs":{"total":96636764160,"available":96574205952,"internally_reserved":0,"allocated":3473408,"data_stored":1799774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159828,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118896,"kb_used_data":59352,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94252944,"statfs":{"total":96636764160,"available":96515014656,"internally_reserved":0,"allocated":60776448,"data_stored":105510880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78884,"kb_used_data":23820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94292956,"statfs":{"total":96636764160,"available":96555986944,"internally_reserved":0,"allocated":24391680,"data_stored":22721203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68824,"kb_used_data":7092,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303016,"statfs":{"total":96636764160,"available":96566288384,"internally_reserved":0,"allocated":7262208,"data_stored":5504749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607607,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113168,"kb_used_data":45880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94258672,"statfs":{"total":96636764160,"available":96520880128,"internally_reserved":0,"allocated":46981120,"data_stored":76126081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640311,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76452,"kb_used_data":18888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295388,"statfs":{"total":96636764160,"available":96558477312,"internally_reserved":0,"allocated":19341312,"data_stored":27654931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3604681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16429056,"data_stored":26422044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":560799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57286656,"data_stored":91004196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41578496,"data_stored":72420601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":556103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:07.170 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:07.222 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:07.219+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:07.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:07.269 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:07.270 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:07.270 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:07.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:50:07.363 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:07.360+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:07.424 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:07.420+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:07.425 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:07.425 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:07.425 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:07.434 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:50:07.459 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-17T21:50:07.497 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:07.498 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:07.510 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-17T21:50:06.798000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308585257,"num_objects":885,"num_object_clones":24,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":723,"num_read":380754,"num_read_kb":1875763,"num_write":22749,"num_write_kb":604706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21082,"ondisk_log_size":21082,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":517316,"kb_used_data":158424,"kb_used_omap":36,"kb_used_meta":356891,"kb_avail":565713724,"statfs":{"total":579820584960,"available":579290853376,"internally_reserved":0,"allocated":162226176,"data_stored":239317618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":365456681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-251303523,"num_objects":-264,"num_object_clones":-80,"num_object_copies":-528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-264,"num_whiteouts":-50,"num_read":1320,"num_read_kb":1121,"num_write":273,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.015451"},"pg_stats":[{"pgid":"3.1f","version":"241'404","reported_seq":11842,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.768819+0000","last_change":"2024-09-17T21:50:03.693789+0000","last_active":"2024-09-17T21:50:03.768819+0000","last_peered":"2024-09-17T21:50:03.768819+0000","last_clean":"2024-09-17T21:50:03.768819+0000","last_became_active":"2024-09-17T21:49:56.760149+0000","last_became_peered":"2024-09-17T21:49:56.760149+0000","last_unstale":"2024-09-17T21:50:03.768819+0000","last_undegraded":"2024-09-17T21:50:03.768819+0000","last_fullsized":"2024-09-17T21:50:03.768819+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":404,"log_dups_size":0,"ondisk_log_size":404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:39.699036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011878212000000001,"stat_sum":{"num_bytes":17712693,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":25,"num_read":11470,"num_read_kb":71057,"num_write":367,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.1e","version":"243'601","reported_seq":20623,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.900065+0000","last_change":"2024-09-17T21:50:05.806530+0000","last_active":"2024-09-17T21:50:05.900065+0000","last_peered":"2024-09-17T21:50:05.900065+0000","last_clean":"2024-09-17T21:50:05.900065+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:05.900065+0000","last_undegraded":"2024-09-17T21:50:05.900065+0000","last_fullsized":"2024-09-17T21:50:05.900065+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:26.800446+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.014483848000000001,"stat_sum":{"num_bytes":17454592,"num_objects":32,"num_object_clones":4,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":26,"num_read":19155,"num_read_kb":109404,"num_write":453,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1d","version":"242'415","reported_seq":5517,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.794366+0000","last_change":"2024-09-17T21:50:05.795063+0000","last_active":"2024-09-17T21:50:05.794366+0000","last_peered":"2024-09-17T21:50:05.794366+0000","last_clean":"2024-09-17T21:50:05.794366+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:05.794366+0000","last_undegraded":"2024-09-17T21:50:05.794366+0000","last_fullsized":"2024-09-17T21:50:05.794366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:57:38.708630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318100000000003,"stat_sum":{"num_bytes":1148416,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":665,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1c","version":"243'656","reported_seq":18601,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.945876+0000","last_change":"2024-09-17T21:50:05.805124+0000","last_active":"2024-09-17T21:50:05.945876+0000","last_peered":"2024-09-17T21:50:05.945876+0000","last_clean":"2024-09-17T21:50:05.945876+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:05.945876+0000","last_undegraded":"2024-09-17T21:50:05.945876+0000","last_fullsized":"2024-09-17T21:50:05.945876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":656,"log_dups_size":0,"ondisk_log_size":656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:32:26.711241+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010758909000000001,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19883,"num_read_kb":87050,"num_write":713,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.1b","version":"242'461","reported_seq":16978,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.977258+0000","last_change":"2024-09-17T21:50:04.797063+0000","last_active":"2024-09-17T21:50:04.977258+0000","last_peered":"2024-09-17T21:50:04.977258+0000","last_clean":"2024-09-17T21:50:04.977258+0000","last_became_active":"2024-09-17T21:49:56.760843+0000","last_became_peered":"2024-09-17T21:49:56.760843+0000","last_unstale":"2024-09-17T21:50:04.977258+0000","last_undegraded":"2024-09-17T21:50:04.977258+0000","last_fullsized":"2024-09-17T21:50:04.977258+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":461,"log_dups_size":0,"ondisk_log_size":461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:46:29.467956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062440200000000003,"stat_sum":{"num_bytes":36542976,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":17,"num_read":14439,"num_read_kb":43518,"num_write":364,"num_write_kb":8466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.1a","version":"242'727","reported_seq":12121,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.980670+0000","last_change":"2024-09-17T21:50:04.797438+0000","last_active":"2024-09-17T21:50:04.980670+0000","last_peered":"2024-09-17T21:50:04.980670+0000","last_clean":"2024-09-17T21:50:04.980670+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:04.980670+0000","last_undegraded":"2024-09-17T21:50:04.980670+0000","last_fullsized":"2024-09-17T21:50:04.980670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":727,"log_dups_size":0,"ondisk_log_size":727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:24:30.039284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097771100000000003,"stat_sum":{"num_bytes":13631594,"num_objects":28,"num_object_clones":1,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":24,"num_read":14100,"num_read_kb":82620,"num_write":737,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.19","version":"241'464","reported_seq":5188,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.723435+0000","last_change":"2024-09-17T21:50:03.724020+0000","last_active":"2024-09-17T21:50:03.723435+0000","last_peered":"2024-09-17T21:50:03.723435+0000","last_clean":"2024-09-17T21:50:03.723435+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:03.723435+0000","last_undegraded":"2024-09-17T21:50:03.723435+0000","last_fullsized":"2024-09-17T21:50:03.723435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:09:21.625796+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0029561269999999998,"stat_sum":{"num_bytes":1949184,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":456,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.18","version":"242'708","reported_seq":13319,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.942423+0000","last_change":"2024-09-17T21:50:05.793650+0000","last_active":"2024-09-17T21:50:05.942423+0000","last_peered":"2024-09-17T21:50:05.942423+0000","last_clean":"2024-09-17T21:50:05.942423+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:05.942423+0000","last_undegraded":"2024-09-17T21:50:05.942423+0000","last_fullsized":"2024-09-17T21:50:05.942423+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:26:37.877005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001869157,"stat_sum":{"num_bytes":24559616,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":22,"num_read":12422,"num_read_kb":69065,"num_write":806,"num_write_kb":13809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.17","version":"242'385","reported_seq":7823,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.800256+0000","last_change":"2024-09-17T21:50:05.801235+0000","last_active":"2024-09-17T21:50:05.800256+0000","last_peered":"2024-09-17T21:50:05.800256+0000","last_clean":"2024-09-17T21:50:05.800256+0000","last_became_active":"2024-09-17T21:49:56.761187+0000","last_became_peered":"2024-09-17T21:49:56.761187+0000","last_unstale":"2024-09-17T21:50:05.800256+0000","last_undegraded":"2024-09-17T21:50:05.800256+0000","last_fullsized":"2024-09-17T21:50:05.800256+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:11:19.656714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0066466060000000002,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.16","version":"242'1024","reported_seq":12045,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.807436+0000","last_change":"2024-09-17T21:50:05.808458+0000","last_active":"2024-09-17T21:50:05.807436+0000","last_peered":"2024-09-17T21:50:05.807436+0000","last_clean":"2024-09-17T21:50:05.807436+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:05.807436+0000","last_undegraded":"2024-09-17T21:50:05.807436+0000","last_fullsized":"2024-09-17T21:50:05.807436+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1024,"log_dups_size":0,"ondisk_log_size":1024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:36:09.441372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01224691,"stat_sum":{"num_bytes":770048,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1044,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.15","version":"240'783","reported_seq":10226,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.693905+0000","last_change":"2024-09-17T21:50:03.695078+0000","last_active":"2024-09-17T21:50:03.693905+0000","last_peered":"2024-09-17T21:50:03.693905+0000","last_clean":"2024-09-17T21:50:03.693905+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:03.693905+0000","last_undegraded":"2024-09-17T21:50:03.693905+0000","last_fullsized":"2024-09-17T21:50:03.693905+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":783,"log_dups_size":0,"ondisk_log_size":783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:07:28.221184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.013107594,"stat_sum":{"num_bytes":2616832,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":729,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.14","version":"240'518","reported_seq":19353,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681132+0000","last_change":"2024-09-17T21:50:03.683218+0000","last_active":"2024-09-17T21:50:03.681132+0000","last_peered":"2024-09-17T21:50:03.681132+0000","last_clean":"2024-09-17T21:50:03.681132+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:03.681132+0000","last_undegraded":"2024-09-17T21:50:03.681132+0000","last_fullsized":"2024-09-17T21:50:03.681132+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:04:08.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028355059999999998,"stat_sum":{"num_bytes":17027088,"num_objects":30,"num_object_clones":1,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":18,"num_read":18243,"num_read_kb":143651,"num_write":560,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.682083+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:03.682083+0000","last_peered":"2024-09-17T21:50:03.682083+0000","last_clean":"2024-09-17T21:50:03.682083+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:03.682083+0000","last_undegraded":"2024-09-17T21:50:03.682083+0000","last_fullsized":"2024-09-17T21:50:03.682083+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"241'749","reported_seq":10817,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.778889+0000","last_change":"2024-09-17T21:50:03.725924+0000","last_active":"2024-09-17T21:50:03.778889+0000","last_peered":"2024-09-17T21:50:03.778889+0000","last_clean":"2024-09-17T21:50:03.778889+0000","last_became_active":"2024-09-17T21:49:56.760413+0000","last_became_peered":"2024-09-17T21:49:56.760413+0000","last_unstale":"2024-09-17T21:50:03.778889+0000","last_undegraded":"2024-09-17T21:50:03.778889+0000","last_fullsized":"2024-09-17T21:50:03.778889+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:25.690688+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0048677659999999999,"stat_sum":{"num_bytes":4534311,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":23,"num_read":12158,"num_read_kb":53859,"num_write":609,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":491,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.720974+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:03.720974+0000","last_peered":"2024-09-17T21:50:03.720974+0000","last_clean":"2024-09-17T21:50:03.720974+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:03.720974+0000","last_undegraded":"2024-09-17T21:50:03.720974+0000","last_fullsized":"2024-09-17T21:50:03.720974+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"242'567","reported_seq":6875,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.791329+0000","last_change":"2024-09-17T21:50:05.792532+0000","last_active":"2024-09-17T21:50:05.791329+0000","last_peered":"2024-09-17T21:50:05.791329+0000","last_clean":"2024-09-17T21:50:05.791329+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:05.791329+0000","last_undegraded":"2024-09-17T21:50:05.791329+0000","last_fullsized":"2024-09-17T21:50:05.791329+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:20.312948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001382643,"stat_sum":{"num_bytes":975872,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681599+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:03.681599+0000","last_peered":"2024-09-17T21:50:03.681599+0000","last_clean":"2024-09-17T21:50:03.681599+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:03.681599+0000","last_undegraded":"2024-09-17T21:50:03.681599+0000","last_fullsized":"2024-09-17T21:50:03.681599+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"241'702","reported_seq":19480,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.726102+0000","last_change":"2024-09-17T21:50:03.726394+0000","last_active":"2024-09-17T21:50:03.726102+0000","last_peered":"2024-09-17T21:50:03.726102+0000","last_clean":"2024-09-17T21:50:03.726102+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:03.726102+0000","last_undegraded":"2024-09-17T21:50:03.726102+0000","last_fullsized":"2024-09-17T21:50:03.726102+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":702,"log_dups_size":0,"ondisk_log_size":702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:19:00.144007+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0051008770000000002,"stat_sum":{"num_bytes":3265536,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":502,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":500,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.720975+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:03.720975+0000","last_peered":"2024-09-17T21:50:03.720975+0000","last_clean":"2024-09-17T21:50:03.720975+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:03.720975+0000","last_undegraded":"2024-09-17T21:50:03.720975+0000","last_fullsized":"2024-09-17T21:50:03.720975+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"242'547","reported_seq":13072,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.925034+0000","last_change":"2024-09-17T21:50:05.800180+0000","last_active":"2024-09-17T21:50:05.925034+0000","last_peered":"2024-09-17T21:50:05.925034+0000","last_clean":"2024-09-17T21:50:05.925034+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:05.925034+0000","last_undegraded":"2024-09-17T21:50:05.925034+0000","last_fullsized":"2024-09-17T21:50:05.925034+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:49:37.894138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057336660000000001,"stat_sum":{"num_bytes":16,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":530,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.3","version":"242'1028","reported_seq":11932,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.949563+0000","last_change":"2024-09-17T21:50:04.797777+0000","last_active":"2024-09-17T21:50:04.949563+0000","last_peered":"2024-09-17T21:50:04.949563+0000","last_clean":"2024-09-17T21:50:04.949563+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:04.949563+0000","last_undegraded":"2024-09-17T21:50:04.949563+0000","last_fullsized":"2024-09-17T21:50:04.949563+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:17:52.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001301394,"stat_sum":{"num_bytes":2097152,"num_objects":19,"num_object_clones":1,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":16,"num_read":14292,"num_read_kb":38832,"num_write":1258,"num_write_kb":10670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"2.2","version":"19'2","reported_seq":471,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.794272+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:05.794272+0000","last_peered":"2024-09-17T21:50:05.794272+0000","last_clean":"2024-09-17T21:50:05.794272+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:05.794272+0000","last_undegraded":"2024-09-17T21:50:05.794272+0000","last_fullsized":"2024-09-17T21:50:05.794272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"241'449","reported_seq":7324,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.728763+0000","last_change":"2024-09-17T21:50:03.728763+0000","last_active":"2024-09-17T21:50:03.728763+0000","last_peered":"2024-09-17T21:50:03.728763+0000","last_clean":"2024-09-17T21:50:03.728763+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:03.728763+0000","last_undegraded":"2024-09-17T21:50:03.728763+0000","last_fullsized":"2024-09-17T21:50:03.728763+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:53:21.265305+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072556410000000002,"stat_sum":{"num_bytes":11170923,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":23,"num_read":10031,"num_read_kb":30206,"num_write":352,"num_write_kb":6762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.790822+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:05.790822+0000","last_peered":"2024-09-17T21:50:05.790822+0000","last_clean":"2024-09-17T21:50:05.790822+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:05.790822+0000","last_undegraded":"2024-09-17T21:50:05.790822+0000","last_fullsized":"2024-09-17T21:50:05.790822+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"242'1890","reported_seq":9198,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.999260+0000","last_change":"2024-09-17T21:50:04.798245+0000","last_active":"2024-09-17T21:50:04.999260+0000","last_peered":"2024-09-17T21:50:04.999260+0000","last_clean":"2024-09-17T21:50:04.999260+0000","last_became_active":"2024-09-17T21:49:56.761242+0000","last_became_peered":"2024-09-17T21:49:56.761242+0000","last_unstale":"2024-09-17T21:50:04.999260+0000","last_undegraded":"2024-09-17T21:50:04.999260+0000","last_fullsized":"2024-09-17T21:50:04.999260+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1890,"log_dups_size":0,"ondisk_log_size":1890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:20:03.394784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001900236,"stat_sum":{"num_bytes":2833003,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":26,"num_read":15736,"num_read_kb":56977,"num_write":2767,"num_write_kb":12581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"2.0","version":"0'0","reported_seq":98,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.679810+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:03.679810+0000","last_peered":"2024-09-17T21:50:03.679810+0000","last_clean":"2024-09-17T21:50:03.679810+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:03.679810+0000","last_undegraded":"2024-09-17T21:50:03.679810+0000","last_fullsized":"2024-09-17T21:50:03.679810+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"241'509","reported_seq":5769,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.726576+0000","last_change":"2024-09-17T21:50:03.726576+0000","last_active":"2024-09-17T21:50:03.726576+0000","last_peered":"2024-09-17T21:50:03.726576+0000","last_clean":"2024-09-17T21:50:03.726576+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:03.726576+0000","last_undegraded":"2024-09-17T21:50:03.726576+0000","last_fullsized":"2024-09-17T21:50:03.726576+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:42:33.439143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052605259999999997,"stat_sum":{"num_bytes":27212816,"num_objects":36,"num_object_clones":3,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":23,"num_read":5949,"num_read_kb":32245,"num_write":561,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"2.3","version":"17'1","reported_seq":488,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.795361+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:05.795361+0000","last_peered":"2024-09-17T21:50:05.795361+0000","last_clean":"2024-09-17T21:50:05.795361+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:05.795361+0000","last_undegraded":"2024-09-17T21:50:05.795361+0000","last_fullsized":"2024-09-17T21:50:05.795361+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":612,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.679847+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:03.679847+0000","last_peered":"2024-09-17T21:50:03.679847+0000","last_clean":"2024-09-17T21:50:03.679847+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:03.679847+0000","last_undegraded":"2024-09-17T21:50:03.679847+0000","last_fullsized":"2024-09-17T21:50:03.679847+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"243'487","reported_seq":7420,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.792886+0000","last_change":"2024-09-17T21:50:05.792886+0000","last_active":"2024-09-17T21:50:05.792886+0000","last_peered":"2024-09-17T21:50:05.792886+0000","last_clean":"2024-09-17T21:50:05.792886+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:05.792886+0000","last_undegraded":"2024-09-17T21:50:05.792886+0000","last_fullsized":"2024-09-17T21:50:05.792886+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:24:41.619178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001132755,"stat_sum":{"num_bytes":12390400,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":24,"num_read":7499,"num_read_kb":58807,"num_write":530,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.9","version":"242'833","reported_seq":11119,"reported_epoch":242,"state":"active+clean","last_fresh":"2024-09-17T21:50:04.891307+0000","last_change":"2024-09-17T21:50:04.798107+0000","last_active":"2024-09-17T21:50:04.891307+0000","last_peered":"2024-09-17T21:50:04.891307+0000","last_clean":"2024-09-17T21:50:04.891307+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:04.891307+0000","last_undegraded":"2024-09-17T21:50:04.891307+0000","last_fullsized":"2024-09-17T21:50:04.891307+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":833,"log_dups_size":0,"ondisk_log_size":833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:30:00.620754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014670569999999999,"stat_sum":{"num_bytes":22810148,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":19,"num_read":15731,"num_read_kb":70297,"num_write":782,"num_write_kb":25464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"6"}]},{"pgid":"3.a","version":"241'537","reported_seq":5388,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.689764+0000","last_change":"2024-09-17T21:50:03.691204+0000","last_active":"2024-09-17T21:50:03.689764+0000","last_peered":"2024-09-17T21:50:03.689764+0000","last_clean":"2024-09-17T21:50:03.689764+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:03.689764+0000","last_undegraded":"2024-09-17T21:50:03.689764+0000","last_fullsized":"2024-09-17T21:50:03.689764+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":537,"log_dups_size":0,"ondisk_log_size":537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:58.687796+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.010337094999999999,"stat_sum":{"num_bytes":921600,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":18,"num_read":4922,"num_read_kb":14304,"num_write":560,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.b","version":"241'584","reported_seq":11022,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.731084+0000","last_change":"2024-09-17T21:50:03.731084+0000","last_active":"2024-09-17T21:50:03.731084+0000","last_peered":"2024-09-17T21:50:03.731084+0000","last_clean":"2024-09-17T21:50:03.731084+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:03.731084+0000","last_undegraded":"2024-09-17T21:50:03.731084+0000","last_fullsized":"2024-09-17T21:50:03.731084+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:53:52.993970+0000","scrub_duration":0,"objects_trimmed":5,"snaptrim_duration":0.0097800590000000007,"stat_sum":{"num_bytes":1601536,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":537,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.c","version":"242'604","reported_seq":19339,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.798643+0000","last_change":"2024-09-17T21:50:05.799346+0000","last_active":"2024-09-17T21:50:05.798643+0000","last_peered":"2024-09-17T21:50:05.798643+0000","last_clean":"2024-09-17T21:50:05.798643+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:05.798643+0000","last_undegraded":"2024-09-17T21:50:05.798643+0000","last_fullsized":"2024-09-17T21:50:05.798643+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:21:53.665692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0049379510000000003,"stat_sum":{"num_bytes":13053991,"num_objects":27,"num_object_clones":1,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":22,"num_read":18226,"num_read_kb":77054,"num_write":590,"num_write_kb":16864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.d","version":"241'854","reported_seq":10109,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.772007+0000","last_change":"2024-09-17T21:50:03.697687+0000","last_active":"2024-09-17T21:50:03.772007+0000","last_peered":"2024-09-17T21:50:03.772007+0000","last_clean":"2024-09-17T21:50:03.772007+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:03.772007+0000","last_undegraded":"2024-09-17T21:50:03.772007+0000","last_fullsized":"2024-09-17T21:50:03.772007+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":854,"log_dups_size":0,"ondisk_log_size":854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:35:18.562267+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015311125,"stat_sum":{"num_bytes":14479398,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":14,"num_read":11772,"num_read_kb":74382,"num_write":929,"num_write_kb":32181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.e","version":"242'436","reported_seq":6108,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.805201+0000","last_change":"2024-09-17T21:50:05.806629+0000","last_active":"2024-09-17T21:50:05.805201+0000","last_peered":"2024-09-17T21:50:05.805201+0000","last_clean":"2024-09-17T21:50:05.805201+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:05.805201+0000","last_undegraded":"2024-09-17T21:50:05.805201+0000","last_fullsized":"2024-09-17T21:50:05.805201+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":436,"log_dups_size":0,"ondisk_log_size":436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:43:43.025624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.010640322000000001,"stat_sum":{"num_bytes":3584000,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":596,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.f","version":"243'1028","reported_seq":17702,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.810367+0000","last_change":"2024-09-17T21:50:05.810367+0000","last_active":"2024-09-17T21:50:05.810367+0000","last_peered":"2024-09-17T21:50:05.810367+0000","last_clean":"2024-09-17T21:50:05.810367+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:05.810367+0000","last_undegraded":"2024-09-17T21:50:05.810367+0000","last_fullsized":"2024-09-17T21:50:05.810367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1028,"log_dups_size":0,"ondisk_log_size":1028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:31:02.255879+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.015530074,"stat_sum":{"num_bytes":442880,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1358,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.10","version":"243'621","reported_seq":7408,"reported_epoch":243,"state":"active+clean","last_fresh":"2024-09-17T21:50:05.798564+0000","last_change":"2024-09-17T21:50:05.798564+0000","last_active":"2024-09-17T21:50:05.798564+0000","last_peered":"2024-09-17T21:50:05.798564+0000","last_clean":"2024-09-17T21:50:05.798564+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:05.798564+0000","last_undegraded":"2024-09-17T21:50:05.798564+0000","last_fullsized":"2024-09-17T21:50:05.798564+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:52:31.274129+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0044640950000000004,"stat_sum":{"num_bytes":1048576,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":640,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"7","length":"7"}]},{"pgid":"3.11","version":"240'544","reported_seq":6376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.681055+0000","last_change":"2024-09-17T21:50:03.682939+0000","last_active":"2024-09-17T21:50:03.681055+0000","last_peered":"2024-09-17T21:50:03.681055+0000","last_clean":"2024-09-17T21:50:03.681055+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:03.681055+0000","last_undegraded":"2024-09-17T21:50:03.681055+0000","last_fullsized":"2024-09-17T21:50:03.681055+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":544,"log_dups_size":0,"ondisk_log_size":544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:41:38.810907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002839084,"stat_sum":{"num_bytes":18665472,"num_objects":32,"num_object_clones":3,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":25,"num_read":4772,"num_read_kb":61222,"num_write":581,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.12","version":"239'387","reported_seq":12689,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.724097+0000","last_change":"2024-09-17T21:50:03.724554+0000","last_active":"2024-09-17T21:50:03.724097+0000","last_peered":"2024-09-17T21:50:03.724097+0000","last_clean":"2024-09-17T21:50:03.724097+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:03.724097+0000","last_undegraded":"2024-09-17T21:50:03.724097+0000","last_fullsized":"2024-09-17T21:50:03.724097+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":387,"log_dups_size":0,"ondisk_log_size":387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:23:53.946554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0033849959999999999,"stat_sum":{"num_bytes":10022912,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":463,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"9","length":"5"}]},{"pgid":"3.13","version":"241'460","reported_seq":10372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-17T21:50:03.692288+0000","last_change":"2024-09-17T21:50:03.693166+0000","last_active":"2024-09-17T21:50:03.692288+0000","last_peered":"2024-09-17T21:50:03.692288+0000","last_clean":"2024-09-17T21:50:03.692288+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:03.692288+0000","last_undegraded":"2024-09-17T21:50:03.692288+0000","last_fullsized":"2024-09-17T21:50:03.692288+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":460,"log_dups_size":0,"ondisk_log_size":460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:12:33.263270+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.012620528000000001,"stat_sum":{"num_bytes":23471289,"num_objects":34,"num_object_clones":2,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":25,"num_read":8737,"num_read_kb":42437,"num_write":558,"num_write_kb":12393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"9","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":307994870,"num_objects":881,"num_object_clones":24,"num_object_copies":1762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":881,"num_whiteouts":723,"num_read":380638,"num_read_kb":1875663,"num_write":22588,"num_write_kb":602288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120020992,"data_stored":194568424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":20962,"ondisk_log_size":20962,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607608,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61092,"kb_used_data":3392,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94310748,"statfs":{"total":96636764160,"available":96574205952,"internally_reserved":0,"allocated":3473408,"data_stored":1799774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159828,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":118896,"kb_used_data":59352,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94252944,"statfs":{"total":96636764160,"available":96515014656,"internally_reserved":0,"allocated":60776448,"data_stored":105510880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607607,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78884,"kb_used_data":23820,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94292956,"statfs":{"total":96636764160,"available":96555986944,"internally_reserved":0,"allocated":24391680,"data_stored":22721203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607607,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68824,"kb_used_data":7092,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303016,"statfs":{"total":96636764160,"available":96566288384,"internally_reserved":0,"allocated":7262208,"data_stored":5504749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607607,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":113168,"kb_used_data":45880,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94258672,"statfs":{"total":96636764160,"available":96520880128,"internally_reserved":0,"allocated":46981120,"data_stored":76126081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640311,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76452,"kb_used_data":18888,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94295388,"statfs":{"total":96636764160,"available":96558477312,"internally_reserved":0,"allocated":19341312,"data_stored":27654931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3661824,"data_stored":3604681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16429056,"data_stored":26422044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":560799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":57286656,"data_stored":91004196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":41578496,"data_stored":72420601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":556103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:07.512 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:50:07.512 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-17T21:50:07.512 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-17T21:50:07.512 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-17T21:50:07.512 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-17T21:50:07.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-17T21:50:07.562 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:07.559+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:07.583 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:50:07.584 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.610703212, 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:50:07.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.610678952, 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:50:07.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:50:07.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:07.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.61065306899999, 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:07.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:50:07.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.61062806800001, 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.599 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:07.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.603 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.610588269, 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.61057252399999, 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:50:07.608 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:50:07.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.61056098899999, 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.613 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:50:07.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:50:07.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:07.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.61055068900001, 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:07.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:50:07.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60857944599999, 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.608566298, 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:50:07.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:50:07.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60855569200001, 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:50:07.631 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:07.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:50:07.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.608545354, 2024-09-17T21:50:07.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:07.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:50:07.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60853534699999, 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:50:07.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:07.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.608525396, 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:07.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.608510088, 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:50:07.650 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:50:07.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60652324500001, 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:50:07.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:07.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60651075600001, 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:50:07.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 230.60650029300001, 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:50:07.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.907164495000004, 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:50:07.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:50:07.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:50:07.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 69.552434519000002, 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:07.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:07.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:07.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:50:07.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:07.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:07.679 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:07.680 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:50:07.680 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:50:07.681 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:50:07.681 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:50:07.682 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:50:07.682 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:50:07.683 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:50:07.683 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:07.683 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-17T21:50:07.683 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:07.683 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:50:07.684 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:50:07.684 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:50:07.685 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:50:07.685 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:07.678+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:07.781 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:07.778+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:07.834 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:07.834 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:07.843 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:50:07.881 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:07.879+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:07.956 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:07.956 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:07.965 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:50:07.983 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:07.981+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:08.072 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.072 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-17T21:50:08.083 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:08.081+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:08.184 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:08.180+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:08.190 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.190 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-17T21:50:08.284 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:08.282+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:08.306 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.306 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-17T21:50:08.385 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:08.383+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:08.422 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.422 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-17T21:50:08.487 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:08.484+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:08.540 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.540 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-17T21:50:08.588 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:08.585+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:08.655 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.655 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-17T21:50:08.708 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-17T21:50:08.706+0000 7fbd645af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32018) UID: 0 2024-09-17T21:50:08.786 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.786 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.789 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:08.787+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:08.795 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-17T21:50:08.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:50:08.890 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:08.887+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:08.946 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:08.946 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:08.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-17T21:50:08.993 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:08.990+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:09.064 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.064 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.073 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-17T21:50:09.117 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:09.115+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:09.181 INFO:teuthology.orchestra.run.smithi007.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.181 INFO:teuthology.orchestra.run.smithi007.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-17T21:50:09.235 INFO:tasks.ceph.osd.4.smithi053.stderr:2024-09-17T21:50:09.231+0000 7f0225a49640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32305) UID: 0 2024-09-17T21:50:09.295 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:09.293+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:09.310 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.310 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.320 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-17T21:50:09.404 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:09.400+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:09.468 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.468 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.477 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-17T21:50:09.517 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:09.515+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:09.598 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-17T21:50:09.595+0000 7fcf2d4e2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 47160) UID: 0 2024-09-17T21:50:09.623 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.623 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-17T21:50:09.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-17T21:50:09.717 INFO:tasks.ceph.osd.5.smithi053.stderr:2024-09-17T21:50:09.714+0000 7fa4c50d2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32308) UID: 0 2024-09-17T21:50:09.780 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-17T21:50:09.780 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-17T21:50:09.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-17T21:50:09.799 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-17T21:50:09.797+0000 7fc475884640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32032) UID: 0 2024-09-17T21:50:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:50:09.819 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0: {} 2024-09-17T21:50:09.819 INFO:teuthology.orchestra.run.smithi007.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-17T21:50:09.837 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1: {} 2024-09-17T21:50:09.837 INFO:teuthology.orchestra.run.smithi007.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-17T21:50:09.855 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2: {} 2024-09-17T21:50:09.855 INFO:teuthology.orchestra.run.smithi007.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-17T21:50:09.873 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3: {} 2024-09-17T21:50:09.873 INFO:teuthology.orchestra.run.smithi007.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-17T21:50:09.889 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4: {} 2024-09-17T21:50:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-17T21:50:09.900 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-17T21:50:09.897+0000 7f7ef013b640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32026) UID: 0 2024-09-17T21:50:09.909 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5: {} 2024-09-17T21:50:09.909 INFO:teuthology.orchestra.run.smithi007.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-17T21:50:09.911 INFO:teuthology.orchestra.run.smithi165.stdout:All operations completed A-OK! 2024-09-17T21:50:09.923 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:09.924 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-17T21:50:09.933 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-17T21:50:09.952 INFO:tasks.thrashosds:joining thrashosds 2024-09-17T21:50:09.954 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:09.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:09.954 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:09.954 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:09.954 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:09.954 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-17T21:50:10.062 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11309 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1142784~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08844995699999, 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068514119999999998, 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11309 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970846+0000", 2024-09-17T21:50:10.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970847+0000", 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5699999999999998e-07 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970849+0000", 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3410000000000001e-06 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970858+0000", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2180000000000005e-06 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970864+0000", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3369999999999999e-06 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039126+0000", 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068262620999999996 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039209+0000", 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2793999999999994e-05 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039361+0000", 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000151352 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.067 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11310 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1191936~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08842569699999, 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068676737000000002, 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.068 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11310 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970871+0000", 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1600000000000002e-07 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970873+0000", 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-06 2024-09-17T21:50:10.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970880+0000", 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4919999999999998e-06 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970885+0000", 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2070000000000003e-06 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039394+0000", 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068509022000000003 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039443+0000", 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9193000000000001e-05 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039547+0000", 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104237 2024-09-17T21:50:10.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11311 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1241088~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08839981400001, 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068924681000000002, 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11311 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970897+0000", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3500000000000002e-07 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970899+0000", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.43e-06 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970908+0000", 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4740000000000005e-06 2024-09-17T21:50:10.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970913+0000", 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5759999999999996e-06 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039581+0000", 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068667984000000001 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039668+0000", 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6050000000000005e-05 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039821+0000", 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015373200000000001 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11312 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1290240~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:10.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.088374813, 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069088250000000004, 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11312 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:10.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970922+0000", 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0200000000000003e-07 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970924+0000", 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970931+0000", 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8109999999999997e-06 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970936+0000", 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.882e-06 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039853+0000", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068916622999999996 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.039895+0000", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2491999999999999e-05 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040010+0000", 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114744 2024-09-17T21:50:10.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11313 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1339392~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08833501399999, 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069326631999999999, 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11313 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970961+0000", 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970962+0000", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7399999999999999e-07 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970963+0000", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.655e-06 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970968+0000", 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-17T21:50:10.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970973+0000", 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5730000000000001e-06 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040046+0000", 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069073183999999996 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040132+0000", 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6607999999999995e-05 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040288+0000", 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015557200000000001 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11314 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1388544~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:10.086 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08831926900001, 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069497348, 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11314 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970977+0000", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2399999999999999e-07 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970979+0000", 2024-09-17T21:50:10.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3209999999999999e-06 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970982+0000", 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2940000000000001e-06 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970985+0000", 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.495e-06 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040321+0000", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069336035000000004 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040370+0000", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8374999999999997e-05 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040474+0000", 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104604 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11315 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1437696~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08830773400001, 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069775755999999994, 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11315 2024-09-17T21:50:10.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970989+0000", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.15e-07 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970990+0000", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.127e-06 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970993+0000", 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.157e-06 2024-09-17T21:50:10.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970996+0000", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6860000000000002e-06 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040510+0000", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069514537000000001 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040597+0000", 2024-09-17T21:50:10.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6677999999999999e-05 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040764+0000", 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016725600000000001 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11316 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1486848~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.088297434, 2024-09-17T21:50:10.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069959146999999999, 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11316 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:10.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.970999+0000", 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972945+0000", 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019462730000000001 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972949+0000", 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1139999999999999e-06 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972957+0000", 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.289e-06 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972961+0000", 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4619999999999998e-06 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040796+0000", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067835281999999997 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040846+0000", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9549999999999998e-05 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040958+0000", 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011217699999999999 2024-09-17T21:50:10.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11317 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1536000~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08632619100001, 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068265177999999996, 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11317 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972970+0000", 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972971+0000", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9499999999999998e-07 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972972+0000", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8640000000000001e-06 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972976+0000", 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8120000000000001e-06 2024-09-17T21:50:10.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972979+0000", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0479999999999999e-06 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.040994+0000", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068014824000000002 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041083+0000", 2024-09-17T21:50:10.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8696000000000006e-05 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041235+0000", 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015253900000000001 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11318 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1585152~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08631304299999, 2024-09-17T21:50:10.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068434964000000001, 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11318 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972983+0000", 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972984+0000", 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4200000000000002e-07 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972985+0000", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.308e-06 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972988+0000", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2140000000000001e-06 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972991+0000", 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5660000000000001e-06 2024-09-17T21:50:10.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041268+0000", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068277811999999993 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041315+0000", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6953000000000003e-05 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041418+0000", 2024-09-17T21:50:10.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010286899999999999 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11319 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1634304~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.086302437, 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068696831999999999, 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.109 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11319 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:10.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972994+0000", 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972996+0000", 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3200000000000001e-06 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.972999+0000", 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0979999999999998e-06 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973001+0000", 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.582e-06 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041454+0000", 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068452536999999994 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041537+0000", 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450999999999998e-05 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041691+0000", 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015361500000000001 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11320 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1683456~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08629209899999, 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068884909999999994, 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11320 2024-09-17T21:50:10.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973004+0000", 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973005+0000", 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973006+0000", 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.176e-06 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973009+0000", 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973011+0000", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5560000000000002e-06 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041725+0000", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068713719000000006 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041772+0000", 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.702e-05 2024-09-17T21:50:10.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041889+0000", 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011721400000000001 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11321 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1732608~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.086282092, 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069152011999999999, 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.118 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11321 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973014+0000", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973015+0000", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1299999999999999e-07 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973016+0000", 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.15e-06 2024-09-17T21:50:10.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973019+0000", 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0769999999999999e-06 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973021+0000", 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4779999999999998e-06 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.041924+0000", 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068902963999999997 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042012+0000", 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7578999999999996e-05 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042166+0000", 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000154551 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11322 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1781760~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08627214099999, 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069332217000000002, 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.123 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11322 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973024+0000", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1400000000000001e-07 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973030+0000", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973033+0000", 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.416e-06 2024-09-17T21:50:10.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973036+0000", 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5809999999999999e-06 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042203+0000", 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069167502000000006 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042252+0000", 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8927000000000001e-05 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042356+0000", 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104168 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11323 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1830912~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:10.127 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08625683299999, 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.069539945000000006, 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11323 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.973040+0000", 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974974+0000", 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019339979999999999 2024-09-17T21:50:10.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974978+0000", 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1999999999999996e-06 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974985+0000", 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2679999999999997e-06 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.974989+0000", 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042392+0000", 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.067402231000000007 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042475+0000", 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3041999999999995e-05 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.042580+0000", 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104897 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11326 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 1978368~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.08426999, 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068336557000000006, 2024-09-17T21:50:10.132 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11326 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975026+0000", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975027+0000", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2000000000000001e-07 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975028+0000", 2024-09-17T21:50:10.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2839999999999999e-06 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975033+0000", 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.899e-06 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975036+0000", 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7889999999999999e-06 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043157+0000", 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068121058999999998 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043250+0000", 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3145000000000004e-05 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043363+0000", 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011306099999999999 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.136 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11327 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2027520~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.084257501, 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068597310999999994, 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.137 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11327 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975039+0000", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5100000000000001e-07 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975040+0000", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.285e-06 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975044+0000", 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2270000000000002e-06 2024-09-17T21:50:10.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975046+0000", 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7e-06 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043397+0000", 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068350452000000006 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043481+0000", 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4132e-05 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043636+0000", 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015526400000000001 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.141 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4481.0:11328 3.1c 3:3d054346:::rbd_data.11125c054388.0000000000000004:7 [read 2076672~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 233.084247038, 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068844087999999998, 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4481", 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/415208039", 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11328 2024-09-17T21:50:10.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975049+0000", 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975050+0000", 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1799999999999999e-07 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975051+0000", 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2699999999999999e-06 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975062+0000", 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0669999999999999e-05 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:16.975064+0000", 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9500000000000001e-06 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043663+0000", 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.068598773000000002 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043752+0000", 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8982999999999998e-05 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:46:17.043893+0000", 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014122399999999999 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:328 3.f 3:f3b0322c:::rbd_data.11125813411b.0000000000000008:head [sparse-read 327680~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.384911239999994, 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99517025000000003, 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 328 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674385+0000", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674386+0000", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8999999999999995e-07 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674391+0000", 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9130000000000002e-06 2024-09-17T21:50:10.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674405+0000", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3837e-05 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674413+0000", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3639999999999999e-06 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674470+0000", 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7315000000000003e-05 2024-09-17T21:50:10.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674479+0000", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5799999999999992e-06 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669324+0000", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99484462399999996 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669449+0000", 2024-09-17T21:50:10.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012536700000000001 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669555+0000", 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010636 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6582.0:161 3.10 3.73c71b0 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e175)", 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:10.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 72.030181264000007, 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63805142000000004, 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6582", 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/26428659", 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 161 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:10.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029115+0000", 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029116+0000", 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7400000000000003e-07 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029119+0000", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6770000000000001e-06 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029128+0000", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.507e-06 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029132+0000", 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5000000000000001e-06 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029177+0000", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4215000000000003e-05 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.029183+0000", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667158+0000", 2024-09-17T21:50:10.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.63797444699999994 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:58.667167+0000", 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0489999999999993e-06 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.158 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:10.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-17T21:50:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:10.269 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:10.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-17T21:50:10.269 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:10.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-17T21:50:10.385 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:10.385 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-17T21:50:10.385 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-17T21:50:10.385 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-17T21:50:10.385 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:10.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.503 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3620 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5627904~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.528004949999996, 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029299032999999999, 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3620 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.504 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971896+0000", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1199999999999999e-07 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971901+0000", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971909+0000", 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5159999999999997e-06 2024-09-17T21:50:10.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971914+0000", 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0649999999999998e-06 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001039+0000", 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029124780999999999 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001124+0000", 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5714e-05 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001195+0000", 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0154999999999995e-05 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3622 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5726208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527980812999999, 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029384839999999999, 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3622 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.509 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971920+0000", 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8299999999999998e-07 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971922+0000", 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.951e-06 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971929+0000", 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2860000000000002e-06 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971938+0000", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.653e-06 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001220+0000", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029282274 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001263+0000", 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2855000000000002e-05 2024-09-17T21:50:10.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001305+0000", 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1437999999999999e-05 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3624 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5824512~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527948558000006, 2024-09-17T21:50:10.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029513409000000001, 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3624 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:10.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971952+0000", 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000001e-07 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971955+0000", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7999999999999999e-06 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971967+0000", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1372999999999999e-05 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971973+0000", 2024-09-17T21:50:10.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1449999999999996e-06 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001329+0000", 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029356107999999999 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001418+0000", 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9221000000000005e-05 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001465+0000", 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7305999999999997e-05 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3626 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5922816~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527918505000002, 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029597641000000001, 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3626 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971982+0000", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4099999999999999e-07 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971985+0000", 2024-09-17T21:50:10.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3719999999999999e-06 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971993+0000", 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0209999999999999e-06 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.971999+0000", 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1419999999999998e-06 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001495+0000", 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029495753999999999 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001536+0000", 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1158000000000001e-05 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001580+0000", 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3752999999999999e-05 2024-09-17T21:50:10.522 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3627 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6021120~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527895158999996, 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029771058, 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3627 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972005+0000", 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972006+0000", 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.96e-07 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972020+0000", 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4408e-05 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972032+0000", 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1392999999999999e-05 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972046+0000", 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001611+0000", 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029565975000000001 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001695+0000", 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3982999999999999e-05 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001776+0000", 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0914000000000005e-05 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.527 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3631 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6119424~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527842211999996, 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029927061000000001, 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3631 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972058+0000", 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972059+0000", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2699999999999999e-07 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972066+0000", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972075+0000", 2024-09-17T21:50:10.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3619999999999995e-06 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972085+0000", 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5370000000000003e-06 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001811+0000", 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029726559999999999 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001903+0000", 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1861999999999996e-05 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.001985+0000", 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2201999999999997e-05 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3632 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6217728~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.527808866000001, 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030073114000000001, 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3632 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972092+0000", 2024-09-17T21:50:10.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4400000000000001e-07 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972094+0000", 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2649999999999999e-06 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972108+0000", 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3281e-05 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.972112+0000", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8799999999999999e-06 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002021+0000", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029908133 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002099+0000", 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8291e-05 2024-09-17T21:50:10.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.002165+0000", 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5820000000000003e-05 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3976 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5218304~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524477633000004, 2024-09-17T21:50:10.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029348506, 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3976 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:10.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975423+0000", 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975424+0000", 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7899999999999997e-07 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975426+0000", 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4980000000000001e-06 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975437+0000", 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1333e-05 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975443+0000", 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8200000000000002e-06 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004614+0000", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029170415000000002 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004689+0000", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5436000000000001e-05 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004771+0000", 2024-09-17T21:50:10.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2225000000000006e-05 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3977 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5316608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524449732999997, 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029521697, 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.542 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3977 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975451+0000", 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2899999999999999e-07 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975453+0000", 2024-09-17T21:50:10.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2069999999999998e-06 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975459+0000", 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370000000000001e-06 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975467+0000", 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3339999999999996e-06 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004805+0000", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029337951000000001 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004880+0000", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5535000000000001e-05 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.004972+0000", 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2304000000000003e-05 2024-09-17T21:50:10.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3978 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5414912~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524427027000002, 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029680775999999999, 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.547 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3978 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975473+0000", 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975474+0000", 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8200000000000001e-07 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975476+0000", 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975482+0000", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.823e-06 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975486+0000", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.284e-06 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005001+0000", 2024-09-17T21:50:10.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029514836999999999 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005075+0000", 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4188000000000002e-05 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005154+0000", 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9140000000000005e-05 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.551 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3979 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5513216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524410759000006, 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029850936000000002, 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.552 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3979 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975490+0000", 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.29e-07 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975491+0000", 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.161e-06 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975494+0000", 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0489999999999999e-06 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975497+0000", 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.475e-06 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005180+0000", 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029683629999999999 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005249+0000", 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8911999999999993e-05 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005341+0000", 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1479999999999998e-05 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3980 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5611520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524400787999994, 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030033250000000001, 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3980 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975500+0000", 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.17e-07 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975501+0000", 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.052e-06 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975506+0000", 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975509+0000", 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7180000000000001e-06 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005370+0000", 2024-09-17T21:50:10.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.029861359000000001 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005444+0000", 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3610999999999997e-05 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005533+0000", 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9040000000000001e-05 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3981 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5709824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524388638999994, 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030176150999999998, 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.562 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3981 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975512+0000", 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2499999999999999e-07 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975513+0000", 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.133e-06 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975516+0000", 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9900000000000002e-06 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975519+0000", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4360000000000001e-06 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005562+0000", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030043534 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005630+0000", 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7910000000000005e-05 2024-09-17T21:50:10.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005688+0000", 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7923000000000001e-05 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3982 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5808128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524378944000006, 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030332795999999999, 2024-09-17T21:50:10.566 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3982 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975522+0000", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975523+0000", 2024-09-17T21:50:10.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.066e-06 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975526+0000", 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9450000000000002e-06 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975528+0000", 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4380000000000002e-06 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005712+0000", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030184276999999999 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005787+0000", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4554999999999993e-05 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005854+0000", 2024-09-17T21:50:10.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7308e-05 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3983 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 5906432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524369351000004, 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030505053000000001, 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3983 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975531+0000", 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0699999999999999e-07 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975532+0000", 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0249999999999999e-06 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975535+0000", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0089999999999999e-06 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975538+0000", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3599999999999999e-06 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005881+0000", 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030343338000000001 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.005968+0000", 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6557999999999994e-05 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006036+0000", 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8555999999999999e-05 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3984 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6004736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524359860999994, 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030655917000000001, 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3984 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975541+0000", 2024-09-17T21:50:10.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0100000000000001e-07 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975542+0000", 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0389999999999999e-06 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975545+0000", 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9830000000000001e-06 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975548+0000", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.723e-06 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006067+0000", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030519568 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006138+0000", 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1169999999999998e-05 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006197+0000", 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8233000000000003e-05 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3985 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6103040~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524350014999996, 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030809968, 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3985 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975550+0000", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975551+0000", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975552+0000", 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1100000000000004e-07 2024-09-17T21:50:10.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975555+0000", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.999e-06 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975557+0000", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3810000000000002e-06 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006222+0000", 2024-09-17T21:50:10.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030665168999999999 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006295+0000", 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2469e-05 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006360+0000", 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5823999999999999e-05 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.5861.0:3986 3.15 3:aeedbb76:::rbd_data.1112c149a998.0000000000000004:34 [read 6201344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e153)", 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 95.524340675000005, 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030969981000000001, 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.5861", 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/220229491", 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 3986 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975560+0000", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1500000000000001e-07 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975561+0000", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8299999999999995e-07 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975564+0000", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1350000000000001e-06 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:34.975568+0000", 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9840000000000003e-06 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006388+0000", 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.030820245 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006465+0000", 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7087000000000002e-05 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:35.006530+0000", 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4331999999999993e-05 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:326 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.826127260999996, 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99508075600000001, 2024-09-17T21:50:10.589 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 326 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673773+0000", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673774+0000", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.266e-06 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673781+0000", 2024-09-17T21:50:10.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7050000000000002e-06 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673873+0000", 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1916000000000006e-05 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673883+0000", 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4339999999999998e-06 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673953+0000", 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0882000000000003e-05 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.673965+0000", 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1963e-05 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668507+0000", 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99454182099999999 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668641+0000", 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013422999999999999 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.668854+0000", 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021253900000000001 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.6427.0:327 3.1e 3:788608a8:::rbd_data.11125813411b.0000000000000007:head [read 331776~61440] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e166)", 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 81.825537288999996, 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99500993400000004, 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "continuous": false, 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.6427", 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.165:0/2467152101", 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 327 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674363+0000", 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674364+0000", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.99e-07 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674368+0000", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.721e-06 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674376+0000", 2024-09-17T21:50:10.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8230000000000001e-06 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674383+0000", 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1380000000000004e-06 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674460+0000", 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7868999999999996e-05 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for peered", 2024-09-17T21:50:10.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:48.674469+0000", 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.551e-06 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669145+0000", 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.99467584099999995 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669186+0000", 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1597000000000002e-05 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-17T21:50:10.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-17T21:48:49.669373+0000", 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018669499999999999 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-17T21:50:10.600 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-17T21:50:11.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-17T21:50:12.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-17T21:50:12.809 INFO:teuthology.orchestra.run.smithi007.stdout:mon.a: {} 2024-09-17T21:50:12.809 INFO:teuthology.orchestra.run.smithi007.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:50:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:noscrub is unset 2024-09-17T21:50:12.821 INFO:teuthology.orchestra.run.smithi007.stdout:mon.b: {} 2024-09-17T21:50:12.821 INFO:teuthology.orchestra.run.smithi007.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:50:12.831 INFO:teuthology.orchestra.run.smithi007.stdout:mon.c: {} 2024-09-17T21:50:12.831 INFO:teuthology.orchestra.run.smithi007.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-17T21:50:12.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-17T21:50:12.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-17T21:50:13.678 INFO:teuthology.orchestra.run.smithi007.stderr:nodeep-scrub is unset 2024-09-17T21:50:13.678 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.0 to 1 (10000) 2024-09-17T21:50:13.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-17T21:50:14.825 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-17T21:50:14.844 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-17T21:50:15.827 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.1 to 1 (10000) 2024-09-17T21:50:15.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-17T21:50:16.837 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-17T21:50:16.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-17T21:50:17.839 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.2 to 1 (10000) 2024-09-17T21:50:17.858 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-17T21:50:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-17T21:50:18.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-17T21:50:19.849 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.4 to 1 (10000) 2024-09-17T21:50:19.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-17T21:50:20.855 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-17T21:50:20.874 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-17T21:50:21.861 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.5 to 1 (10000) 2024-09-17T21:50:21.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-17T21:50:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-17T21:50:22.886 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-17T21:50:23.876 INFO:teuthology.orchestra.run.smithi007.stderr:reweighted osd.3 to 1 (10000) 2024-09-17T21:50:23.893 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-17T21:50:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-17T21:50:24.896 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-17T21:50:24.896 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-17T21:50:24.896 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-17T21:50:24.896 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-17T21:50:24.897 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T21:50:24.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:50:25.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:25.192 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":260,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:50:24.874663+0000","last_up_change":"2024-09-17T21:48:29.752823+0000","last_in_change":"2024-09-17T21:49:18.682438+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:45:50.763887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:46:00.113604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":57,"snap_epoch":247,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":250,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":95,"up_from":148,"up_thru":233,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6817","nonce":51640907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6819","nonce":51640907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6823","nonce":51640907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6821","nonce":51640907}]},"public_addr":"172.21.15.53:6817/51640907","cluster_addr":"172.21.15.53:6819/51640907","heartbeat_back_addr":"172.21.15.53:6823/51640907","heartbeat_front_addr":"172.21.15.53:6821/51640907","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":97},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[1,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:50:25.205 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T21:50:25.205 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:50:25.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:25.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":260,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:50:24.874663+0000","last_up_change":"2024-09-17T21:48:29.752823+0000","last_in_change":"2024-09-17T21:49:18.682438+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:45:50.763887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:46:00.113604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":57,"snap_epoch":247,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":250,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":95,"up_from":148,"up_thru":233,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6817","nonce":51640907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6819","nonce":51640907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6823","nonce":51640907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6821","nonce":51640907}]},"public_addr":"172.21.15.53:6817/51640907","cluster_addr":"172.21.15.53:6819/51640907","heartbeat_back_addr":"172.21.15.53:6823/51640907","heartbeat_front_addr":"172.21.15.53:6821/51640907","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":97},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[1,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:50:25.507 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T21:50:25.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T21:50:25.508 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T21:50:25.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T21:50:25.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T21:50:25.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T21:50:25.667 INFO:teuthology.orchestra.run.smithi007.stdout:635655159833 2024-09-17T21:50:25.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:50:25.669 INFO:teuthology.orchestra.run.smithi007.stdout:51539607612 2024-09-17T21:50:25.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:50:25.715 INFO:teuthology.orchestra.run.smithi007.stdout:51539607611 2024-09-17T21:50:25.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:50:25.716 INFO:teuthology.orchestra.run.smithi007.stdout:51539607612 2024-09-17T21:50:25.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:50:25.727 INFO:teuthology.orchestra.run.smithi007.stdout:47244640316 2024-09-17T21:50:25.727 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:50:25.729 INFO:teuthology.orchestra.run.smithi007.stdout:51539607611 2024-09-17T21:50:25.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:50:26.039 INFO:teuthology.orchestra.run.smithi007.stdout:51539607610 2024-09-17T21:50:26.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607612 got 51539607610 for osd.4 2024-09-17T21:50:26.119 INFO:teuthology.orchestra.run.smithi007.stdout:47244640315 2024-09-17T21:50:26.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640316 got 47244640315 for osd.1 2024-09-17T21:50:26.181 INFO:teuthology.orchestra.run.smithi007.stdout:51539607610 2024-09-17T21:50:26.187 INFO:teuthology.orchestra.run.smithi007.stdout:635655159832 2024-09-17T21:50:26.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607611 got 51539607610 for osd.2 2024-09-17T21:50:26.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 635655159833 got 635655159832 for osd.3 2024-09-17T21:50:26.224 INFO:teuthology.orchestra.run.smithi007.stdout:51539607610 2024-09-17T21:50:26.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607611 got 51539607610 for osd.0 2024-09-17T21:50:26.253 INFO:teuthology.orchestra.run.smithi007.stdout:51539607611 2024-09-17T21:50:26.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607612 got 51539607611 for osd.5 2024-09-17T21:50:27.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T21:50:27.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T21:50:27.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T21:50:27.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T21:50:27.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T21:50:27.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T21:50:27.408 INFO:teuthology.orchestra.run.smithi007.stdout:51539607612 2024-09-17T21:50:27.421 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607612 got 51539607612 for osd.4 2024-09-17T21:50:27.422 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.569 INFO:teuthology.orchestra.run.smithi007.stdout:51539607611 2024-09-17T21:50:27.577 INFO:teuthology.orchestra.run.smithi007.stdout:635655159833 2024-09-17T21:50:27.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607611 got 51539607611 for osd.0 2024-09-17T21:50:27.583 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 635655159833 got 635655159833 for osd.3 2024-09-17T21:50:27.591 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.628 INFO:teuthology.orchestra.run.smithi007.stdout:51539607613 2024-09-17T21:50:27.633 INFO:teuthology.orchestra.run.smithi007.stdout:47244640316 2024-09-17T21:50:27.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607612 got 51539607613 for osd.5 2024-09-17T21:50:27.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640316 got 47244640316 for osd.1 2024-09-17T21:50:27.647 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.648 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.689 INFO:teuthology.orchestra.run.smithi007.stdout:51539607612 2024-09-17T21:50:27.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 51539607611 got 51539607612 for osd.2 2024-09-17T21:50:27.703 DEBUG:teuthology.parallel:result is None 2024-09-17T21:50:27.703 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-17T21:50:27.703 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:28.032 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:28.033 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:28.044 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-17T21:50:26.148329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":359988,"kb_used_data":19548,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871052,"statfs":{"total":579820584960,"available":579451957248,"internally_reserved":0,"allocated":20017152,"data_stored":9146754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.287900"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931666+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:23.931666+0000","last_peered":"2024-09-17T21:50:23.931666+0000","last_clean":"2024-09-17T21:50:23.931666+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:23.931666+0000","last_undegraded":"2024-09-17T21:50:23.931666+0000","last_fullsized":"2024-09-17T21:50:23.931666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5259,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931625+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:23.931625+0000","last_peered":"2024-09-17T21:50:23.931625+0000","last_clean":"2024-09-17T21:50:23.931625+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:23.931625+0000","last_undegraded":"2024-09-17T21:50:23.931625+0000","last_fullsized":"2024-09-17T21:50:23.931625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931905+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:23.931905+0000","last_peered":"2024-09-17T21:50:23.931905+0000","last_clean":"2024-09-17T21:50:23.931905+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:23.931905+0000","last_undegraded":"2024-09-17T21:50:23.931905+0000","last_fullsized":"2024-09-17T21:50:23.931905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19554,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931544+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:23.931544+0000","last_peered":"2024-09-17T21:50:23.931544+0000","last_clean":"2024-09-17T21:50:23.931544+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:23.931544+0000","last_undegraded":"2024-09-17T21:50:23.931544+0000","last_fullsized":"2024-09-17T21:50:23.931544+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":536,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931804+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:23.931804+0000","last_peered":"2024-09-17T21:50:23.931804+0000","last_clean":"2024-09-17T21:50:23.931804+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:23.931804+0000","last_undegraded":"2024-09-17T21:50:23.931804+0000","last_fullsized":"2024-09-17T21:50:23.931804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12128,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931921+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:23.931921+0000","last_peered":"2024-09-17T21:50:23.931921+0000","last_clean":"2024-09-17T21:50:23.931921+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:23.931921+0000","last_undegraded":"2024-09-17T21:50:23.931921+0000","last_fullsized":"2024-09-17T21:50:23.931921+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7532,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931531+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:23.931531+0000","last_peered":"2024-09-17T21:50:23.931531+0000","last_clean":"2024-09-17T21:50:23.931531+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:23.931531+0000","last_undegraded":"2024-09-17T21:50:23.931531+0000","last_fullsized":"2024-09-17T21:50:23.931531+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5853,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931798+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:23.931798+0000","last_peered":"2024-09-17T21:50:23.931798+0000","last_clean":"2024-09-17T21:50:23.931798+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:23.931798+0000","last_undegraded":"2024-09-17T21:50:23.931798+0000","last_fullsized":"2024-09-17T21:50:23.931798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11200,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931800+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:23.931800+0000","last_peered":"2024-09-17T21:50:23.931800+0000","last_clean":"2024-09-17T21:50:23.931800+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:23.931800+0000","last_undegraded":"2024-09-17T21:50:23.931800+0000","last_fullsized":"2024-09-17T21:50:23.931800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11093,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931763+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:23.931763+0000","last_peered":"2024-09-17T21:50:23.931763+0000","last_clean":"2024-09-17T21:50:23.931763+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:23.931763+0000","last_undegraded":"2024-09-17T21:50:23.931763+0000","last_fullsized":"2024-09-17T21:50:23.931763+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19413,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851534+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851534+0000","last_peered":"2024-09-17T21:50:25.851534+0000","last_clean":"2024-09-17T21:50:25.851534+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851534+0000","last_undegraded":"2024-09-17T21:50:25.851534+0000","last_fullsized":"2024-09-17T21:50:25.851534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851631+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851631+0000","last_peered":"2024-09-17T21:50:25.851631+0000","last_clean":"2024-09-17T21:50:25.851631+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851631+0000","last_undegraded":"2024-09-17T21:50:25.851631+0000","last_fullsized":"2024-09-17T21:50:25.851631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851656+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851656+0000","last_peered":"2024-09-17T21:50:25.851656+0000","last_clean":"2024-09-17T21:50:25.851656+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851656+0000","last_undegraded":"2024-09-17T21:50:25.851656+0000","last_fullsized":"2024-09-17T21:50:25.851656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12758,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931667+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:23.931667+0000","last_peered":"2024-09-17T21:50:23.931667+0000","last_clean":"2024-09-17T21:50:23.931667+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:23.931667+0000","last_undegraded":"2024-09-17T21:50:23.931667+0000","last_fullsized":"2024-09-17T21:50:23.931667+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60620,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311220,"statfs":{"total":96636764160,"available":96574689280,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607612,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607611,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607612,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70348,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301492,"statfs":{"total":96636764160,"available":96564727808,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":3064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":3137536,"data_stored":1325210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:28.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:28.337 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:28.349 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T21:50:28.148740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":359988,"kb_used_data":19548,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871052,"statfs":{"total":579820584960,"available":579451957248,"internally_reserved":0,"allocated":20017152,"data_stored":9146754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001300"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931666+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:23.931666+0000","last_peered":"2024-09-17T21:50:23.931666+0000","last_clean":"2024-09-17T21:50:23.931666+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:23.931666+0000","last_undegraded":"2024-09-17T21:50:23.931666+0000","last_fullsized":"2024-09-17T21:50:23.931666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5259,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931625+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:23.931625+0000","last_peered":"2024-09-17T21:50:23.931625+0000","last_clean":"2024-09-17T21:50:23.931625+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:23.931625+0000","last_undegraded":"2024-09-17T21:50:23.931625+0000","last_fullsized":"2024-09-17T21:50:23.931625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931905+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:23.931905+0000","last_peered":"2024-09-17T21:50:23.931905+0000","last_clean":"2024-09-17T21:50:23.931905+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:23.931905+0000","last_undegraded":"2024-09-17T21:50:23.931905+0000","last_fullsized":"2024-09-17T21:50:23.931905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19554,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931544+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:23.931544+0000","last_peered":"2024-09-17T21:50:23.931544+0000","last_clean":"2024-09-17T21:50:23.931544+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:23.931544+0000","last_undegraded":"2024-09-17T21:50:23.931544+0000","last_fullsized":"2024-09-17T21:50:23.931544+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":536,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931804+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:23.931804+0000","last_peered":"2024-09-17T21:50:23.931804+0000","last_clean":"2024-09-17T21:50:23.931804+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:23.931804+0000","last_undegraded":"2024-09-17T21:50:23.931804+0000","last_fullsized":"2024-09-17T21:50:23.931804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12128,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931921+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:23.931921+0000","last_peered":"2024-09-17T21:50:23.931921+0000","last_clean":"2024-09-17T21:50:23.931921+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:23.931921+0000","last_undegraded":"2024-09-17T21:50:23.931921+0000","last_fullsized":"2024-09-17T21:50:23.931921+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7532,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931531+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:23.931531+0000","last_peered":"2024-09-17T21:50:23.931531+0000","last_clean":"2024-09-17T21:50:23.931531+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:23.931531+0000","last_undegraded":"2024-09-17T21:50:23.931531+0000","last_fullsized":"2024-09-17T21:50:23.931531+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5853,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931798+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:23.931798+0000","last_peered":"2024-09-17T21:50:23.931798+0000","last_clean":"2024-09-17T21:50:23.931798+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:23.931798+0000","last_undegraded":"2024-09-17T21:50:23.931798+0000","last_fullsized":"2024-09-17T21:50:23.931798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11200,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931800+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:23.931800+0000","last_peered":"2024-09-17T21:50:23.931800+0000","last_clean":"2024-09-17T21:50:23.931800+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:23.931800+0000","last_undegraded":"2024-09-17T21:50:23.931800+0000","last_fullsized":"2024-09-17T21:50:23.931800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11093,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931763+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:23.931763+0000","last_peered":"2024-09-17T21:50:23.931763+0000","last_clean":"2024-09-17T21:50:23.931763+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:23.931763+0000","last_undegraded":"2024-09-17T21:50:23.931763+0000","last_fullsized":"2024-09-17T21:50:23.931763+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19413,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851534+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851534+0000","last_peered":"2024-09-17T21:50:25.851534+0000","last_clean":"2024-09-17T21:50:25.851534+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851534+0000","last_undegraded":"2024-09-17T21:50:25.851534+0000","last_fullsized":"2024-09-17T21:50:25.851534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851631+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851631+0000","last_peered":"2024-09-17T21:50:25.851631+0000","last_clean":"2024-09-17T21:50:25.851631+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851631+0000","last_undegraded":"2024-09-17T21:50:25.851631+0000","last_fullsized":"2024-09-17T21:50:25.851631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851656+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851656+0000","last_peered":"2024-09-17T21:50:25.851656+0000","last_clean":"2024-09-17T21:50:25.851656+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851656+0000","last_undegraded":"2024-09-17T21:50:25.851656+0000","last_fullsized":"2024-09-17T21:50:25.851656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12758,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931667+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:23.931667+0000","last_peered":"2024-09-17T21:50:23.931667+0000","last_clean":"2024-09-17T21:50:23.931667+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:23.931667+0000","last_undegraded":"2024-09-17T21:50:23.931667+0000","last_fullsized":"2024-09-17T21:50:23.931667+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60620,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311220,"statfs":{"total":96636764160,"available":96574689280,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607612,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607612,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607612,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70348,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301492,"statfs":{"total":96636764160,"available":96564727808,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":3064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":3137536,"data_stored":1325210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:28.351 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-17T21:50:28.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-17T21:50:28.657 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T21:50:28.668 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T21:50:28.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:28.958 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:28.959 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T21:50:28.148740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":359988,"kb_used_data":19548,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871052,"statfs":{"total":579820584960,"available":579451957248,"internally_reserved":0,"allocated":20017152,"data_stored":9146754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001300"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931666+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:23.931666+0000","last_peered":"2024-09-17T21:50:23.931666+0000","last_clean":"2024-09-17T21:50:23.931666+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:23.931666+0000","last_undegraded":"2024-09-17T21:50:23.931666+0000","last_fullsized":"2024-09-17T21:50:23.931666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5259,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931625+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:23.931625+0000","last_peered":"2024-09-17T21:50:23.931625+0000","last_clean":"2024-09-17T21:50:23.931625+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:23.931625+0000","last_undegraded":"2024-09-17T21:50:23.931625+0000","last_fullsized":"2024-09-17T21:50:23.931625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931905+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:23.931905+0000","last_peered":"2024-09-17T21:50:23.931905+0000","last_clean":"2024-09-17T21:50:23.931905+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:23.931905+0000","last_undegraded":"2024-09-17T21:50:23.931905+0000","last_fullsized":"2024-09-17T21:50:23.931905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19554,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931544+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:23.931544+0000","last_peered":"2024-09-17T21:50:23.931544+0000","last_clean":"2024-09-17T21:50:23.931544+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:23.931544+0000","last_undegraded":"2024-09-17T21:50:23.931544+0000","last_fullsized":"2024-09-17T21:50:23.931544+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":536,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931804+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:23.931804+0000","last_peered":"2024-09-17T21:50:23.931804+0000","last_clean":"2024-09-17T21:50:23.931804+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:23.931804+0000","last_undegraded":"2024-09-17T21:50:23.931804+0000","last_fullsized":"2024-09-17T21:50:23.931804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12128,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931921+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:23.931921+0000","last_peered":"2024-09-17T21:50:23.931921+0000","last_clean":"2024-09-17T21:50:23.931921+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:23.931921+0000","last_undegraded":"2024-09-17T21:50:23.931921+0000","last_fullsized":"2024-09-17T21:50:23.931921+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7532,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931531+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:23.931531+0000","last_peered":"2024-09-17T21:50:23.931531+0000","last_clean":"2024-09-17T21:50:23.931531+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:23.931531+0000","last_undegraded":"2024-09-17T21:50:23.931531+0000","last_fullsized":"2024-09-17T21:50:23.931531+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5853,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931798+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:23.931798+0000","last_peered":"2024-09-17T21:50:23.931798+0000","last_clean":"2024-09-17T21:50:23.931798+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:23.931798+0000","last_undegraded":"2024-09-17T21:50:23.931798+0000","last_fullsized":"2024-09-17T21:50:23.931798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11200,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931800+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:23.931800+0000","last_peered":"2024-09-17T21:50:23.931800+0000","last_clean":"2024-09-17T21:50:23.931800+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:23.931800+0000","last_undegraded":"2024-09-17T21:50:23.931800+0000","last_fullsized":"2024-09-17T21:50:23.931800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11093,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931763+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:23.931763+0000","last_peered":"2024-09-17T21:50:23.931763+0000","last_clean":"2024-09-17T21:50:23.931763+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:23.931763+0000","last_undegraded":"2024-09-17T21:50:23.931763+0000","last_fullsized":"2024-09-17T21:50:23.931763+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19413,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851534+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851534+0000","last_peered":"2024-09-17T21:50:25.851534+0000","last_clean":"2024-09-17T21:50:25.851534+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851534+0000","last_undegraded":"2024-09-17T21:50:25.851534+0000","last_fullsized":"2024-09-17T21:50:25.851534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851631+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851631+0000","last_peered":"2024-09-17T21:50:25.851631+0000","last_clean":"2024-09-17T21:50:25.851631+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851631+0000","last_undegraded":"2024-09-17T21:50:25.851631+0000","last_fullsized":"2024-09-17T21:50:25.851631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851656+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851656+0000","last_peered":"2024-09-17T21:50:25.851656+0000","last_clean":"2024-09-17T21:50:25.851656+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851656+0000","last_undegraded":"2024-09-17T21:50:25.851656+0000","last_fullsized":"2024-09-17T21:50:25.851656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12758,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931667+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:23.931667+0000","last_peered":"2024-09-17T21:50:23.931667+0000","last_clean":"2024-09-17T21:50:23.931667+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:23.931667+0000","last_undegraded":"2024-09-17T21:50:23.931667+0000","last_fullsized":"2024-09-17T21:50:23.931667+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60620,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311220,"statfs":{"total":96636764160,"available":96574689280,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607612,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607612,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607612,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70348,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301492,"statfs":{"total":96636764160,"available":96564727808,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":3064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":3137536,"data_stored":1325210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:28.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:29.260 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:29.261 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:29.273 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T21:50:28.148740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":359988,"kb_used_data":19548,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871052,"statfs":{"total":579820584960,"available":579451957248,"internally_reserved":0,"allocated":20017152,"data_stored":9146754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001300"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931666+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:23.931666+0000","last_peered":"2024-09-17T21:50:23.931666+0000","last_clean":"2024-09-17T21:50:23.931666+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:23.931666+0000","last_undegraded":"2024-09-17T21:50:23.931666+0000","last_fullsized":"2024-09-17T21:50:23.931666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5259,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931625+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:23.931625+0000","last_peered":"2024-09-17T21:50:23.931625+0000","last_clean":"2024-09-17T21:50:23.931625+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:23.931625+0000","last_undegraded":"2024-09-17T21:50:23.931625+0000","last_fullsized":"2024-09-17T21:50:23.931625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931905+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:23.931905+0000","last_peered":"2024-09-17T21:50:23.931905+0000","last_clean":"2024-09-17T21:50:23.931905+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:23.931905+0000","last_undegraded":"2024-09-17T21:50:23.931905+0000","last_fullsized":"2024-09-17T21:50:23.931905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19554,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931544+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:23.931544+0000","last_peered":"2024-09-17T21:50:23.931544+0000","last_clean":"2024-09-17T21:50:23.931544+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:23.931544+0000","last_undegraded":"2024-09-17T21:50:23.931544+0000","last_fullsized":"2024-09-17T21:50:23.931544+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":536,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931804+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:23.931804+0000","last_peered":"2024-09-17T21:50:23.931804+0000","last_clean":"2024-09-17T21:50:23.931804+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:23.931804+0000","last_undegraded":"2024-09-17T21:50:23.931804+0000","last_fullsized":"2024-09-17T21:50:23.931804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12128,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931921+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:23.931921+0000","last_peered":"2024-09-17T21:50:23.931921+0000","last_clean":"2024-09-17T21:50:23.931921+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:23.931921+0000","last_undegraded":"2024-09-17T21:50:23.931921+0000","last_fullsized":"2024-09-17T21:50:23.931921+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7532,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931531+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:23.931531+0000","last_peered":"2024-09-17T21:50:23.931531+0000","last_clean":"2024-09-17T21:50:23.931531+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:23.931531+0000","last_undegraded":"2024-09-17T21:50:23.931531+0000","last_fullsized":"2024-09-17T21:50:23.931531+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5853,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931798+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:23.931798+0000","last_peered":"2024-09-17T21:50:23.931798+0000","last_clean":"2024-09-17T21:50:23.931798+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:23.931798+0000","last_undegraded":"2024-09-17T21:50:23.931798+0000","last_fullsized":"2024-09-17T21:50:23.931798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11200,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931800+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:23.931800+0000","last_peered":"2024-09-17T21:50:23.931800+0000","last_clean":"2024-09-17T21:50:23.931800+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:23.931800+0000","last_undegraded":"2024-09-17T21:50:23.931800+0000","last_fullsized":"2024-09-17T21:50:23.931800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11093,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931763+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:23.931763+0000","last_peered":"2024-09-17T21:50:23.931763+0000","last_clean":"2024-09-17T21:50:23.931763+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:23.931763+0000","last_undegraded":"2024-09-17T21:50:23.931763+0000","last_fullsized":"2024-09-17T21:50:23.931763+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19413,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851534+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851534+0000","last_peered":"2024-09-17T21:50:25.851534+0000","last_clean":"2024-09-17T21:50:25.851534+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851534+0000","last_undegraded":"2024-09-17T21:50:25.851534+0000","last_fullsized":"2024-09-17T21:50:25.851534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851631+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851631+0000","last_peered":"2024-09-17T21:50:25.851631+0000","last_clean":"2024-09-17T21:50:25.851631+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851631+0000","last_undegraded":"2024-09-17T21:50:25.851631+0000","last_fullsized":"2024-09-17T21:50:25.851631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851656+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851656+0000","last_peered":"2024-09-17T21:50:25.851656+0000","last_clean":"2024-09-17T21:50:25.851656+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851656+0000","last_undegraded":"2024-09-17T21:50:25.851656+0000","last_fullsized":"2024-09-17T21:50:25.851656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12758,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931667+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:23.931667+0000","last_peered":"2024-09-17T21:50:23.931667+0000","last_clean":"2024-09-17T21:50:23.931667+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:23.931667+0000","last_undegraded":"2024-09-17T21:50:23.931667+0000","last_fullsized":"2024-09-17T21:50:23.931667+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60620,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311220,"statfs":{"total":96636764160,"available":96574689280,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607612,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607612,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607612,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70348,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301492,"statfs":{"total":96636764160,"available":96564727808,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":3064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":3137536,"data_stored":1325210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:29.274 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T21:50:29.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:29.565 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:29.577 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-17T21:50:28.148740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":359988,"kb_used_data":19548,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871052,"statfs":{"total":579820584960,"available":579451957248,"internally_reserved":0,"allocated":20017152,"data_stored":9146754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001300"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12219,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931666+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:23.931666+0000","last_peered":"2024-09-17T21:50:23.931666+0000","last_clean":"2024-09-17T21:50:23.931666+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:23.931666+0000","last_undegraded":"2024-09-17T21:50:23.931666+0000","last_fullsized":"2024-09-17T21:50:23.931666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5259,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931625+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:23.931625+0000","last_peered":"2024-09-17T21:50:23.931625+0000","last_clean":"2024-09-17T21:50:23.931625+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:23.931625+0000","last_undegraded":"2024-09-17T21:50:23.931625+0000","last_fullsized":"2024-09-17T21:50:23.931625+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":528,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931905+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:23.931905+0000","last_peered":"2024-09-17T21:50:23.931905+0000","last_clean":"2024-09-17T21:50:23.931905+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:23.931905+0000","last_undegraded":"2024-09-17T21:50:23.931905+0000","last_fullsized":"2024-09-17T21:50:23.931905+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19554,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931544+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:23.931544+0000","last_peered":"2024-09-17T21:50:23.931544+0000","last_clean":"2024-09-17T21:50:23.931544+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:23.931544+0000","last_undegraded":"2024-09-17T21:50:23.931544+0000","last_fullsized":"2024-09-17T21:50:23.931544+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":536,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931804+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:23.931804+0000","last_peered":"2024-09-17T21:50:23.931804+0000","last_clean":"2024-09-17T21:50:23.931804+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:23.931804+0000","last_undegraded":"2024-09-17T21:50:23.931804+0000","last_fullsized":"2024-09-17T21:50:23.931804+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12128,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931921+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:23.931921+0000","last_peered":"2024-09-17T21:50:23.931921+0000","last_clean":"2024-09-17T21:50:23.931921+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:23.931921+0000","last_undegraded":"2024-09-17T21:50:23.931921+0000","last_fullsized":"2024-09-17T21:50:23.931921+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7532,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931531+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:23.931531+0000","last_peered":"2024-09-17T21:50:23.931531+0000","last_clean":"2024-09-17T21:50:23.931531+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:23.931531+0000","last_undegraded":"2024-09-17T21:50:23.931531+0000","last_fullsized":"2024-09-17T21:50:23.931531+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5853,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931798+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:23.931798+0000","last_peered":"2024-09-17T21:50:23.931798+0000","last_clean":"2024-09-17T21:50:23.931798+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:23.931798+0000","last_undegraded":"2024-09-17T21:50:23.931798+0000","last_fullsized":"2024-09-17T21:50:23.931798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11200,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931800+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:23.931800+0000","last_peered":"2024-09-17T21:50:23.931800+0000","last_clean":"2024-09-17T21:50:23.931800+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:23.931800+0000","last_undegraded":"2024-09-17T21:50:23.931800+0000","last_fullsized":"2024-09-17T21:50:23.931800+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11093,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931763+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:23.931763+0000","last_peered":"2024-09-17T21:50:23.931763+0000","last_clean":"2024-09-17T21:50:23.931763+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:23.931763+0000","last_undegraded":"2024-09-17T21:50:23.931763+0000","last_fullsized":"2024-09-17T21:50:23.931763+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19413,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851534+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851534+0000","last_peered":"2024-09-17T21:50:25.851534+0000","last_clean":"2024-09-17T21:50:25.851534+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851534+0000","last_undegraded":"2024-09-17T21:50:25.851534+0000","last_fullsized":"2024-09-17T21:50:25.851534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6162,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851631+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851631+0000","last_peered":"2024-09-17T21:50:25.851631+0000","last_clean":"2024-09-17T21:50:25.851631+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851631+0000","last_undegraded":"2024-09-17T21:50:25.851631+0000","last_fullsized":"2024-09-17T21:50:25.851631+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17756,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851656+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851656+0000","last_peered":"2024-09-17T21:50:25.851656+0000","last_clean":"2024-09-17T21:50:25.851656+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851656+0000","last_undegraded":"2024-09-17T21:50:25.851656+0000","last_fullsized":"2024-09-17T21:50:25.851656+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12758,"reported_epoch":259,"state":"active+clean","last_fresh":"2024-09-17T21:50:23.931667+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:23.931667+0000","last_peered":"2024-09-17T21:50:23.931667+0000","last_clean":"2024-09-17T21:50:23.931667+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:23.931667+0000","last_undegraded":"2024-09-17T21:50:23.931667+0000","last_fullsized":"2024-09-17T21:50:23.931667+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60620,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311220,"statfs":{"total":96636764160,"available":96574689280,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159833,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607612,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607612,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607612,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70348,"kb_used_data":3060,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301492,"statfs":{"total":96636764160,"available":96564727808,"internally_reserved":0,"allocated":3133440,"data_stored":1325191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640316,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60624,"kb_used_data":3064,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311216,"statfs":{"total":96636764160,"available":96574685184,"internally_reserved":0,"allocated":3137536,"data_stored":1325210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:29.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T21:50:29.867 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:29.868 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":260,"fsid":"195a3125-addb-4952-bbe0-175c3f942cb7","created":"2024-09-17T21:45:33.855733+0000","modified":"2024-09-17T21:50:24.874663+0000","last_up_change":"2024-09-17T21:48:29.752823+0000","last_in_change":"2024-09-17T21:49:18.682438+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T21:45:46.146319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-17T21:45:50.763887+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-17T21:46:00.113604+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"247","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":57,"snap_epoch":247,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2e180d1f-2ad9-4830-8725-c22ba929de39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":250,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6817","nonce":3113291974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6819","nonce":3113291974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6823","nonce":3113291974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":3113291974},{"type":"v1","addr":"172.21.15.7:6821","nonce":3113291974}]},"public_addr":"172.21.15.7:6817/3113291974","cluster_addr":"172.21.15.7:6819/3113291974","heartbeat_back_addr":"172.21.15.7:6823/3113291974","heartbeat_front_addr":"172.21.15.7:6821/3113291974","state":["exists","up"]},{"osd":1,"uuid":"4d534322-006a-4257-b243-601aa8088f95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6801","nonce":2863076940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6803","nonce":2863076940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6807","nonce":2863076940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2863076940},{"type":"v1","addr":"172.21.15.7:6805","nonce":2863076940}]},"public_addr":"172.21.15.7:6801/2863076940","cluster_addr":"172.21.15.7:6803/2863076940","heartbeat_back_addr":"172.21.15.7:6807/2863076940","heartbeat_front_addr":"172.21.15.7:6805/2863076940","state":["exists","up"]},{"osd":2,"uuid":"c0a8a3ad-097f-43cd-8cd3-3f3e6e2b5298","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6809","nonce":3631167928}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6811","nonce":3631167928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6815","nonce":3631167928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":3631167928},{"type":"v1","addr":"172.21.15.7:6813","nonce":3631167928}]},"public_addr":"172.21.15.7:6809/3631167928","cluster_addr":"172.21.15.7:6811/3631167928","heartbeat_back_addr":"172.21.15.7:6815/3631167928","heartbeat_front_addr":"172.21.15.7:6813/3631167928","state":["exists","up"]},{"osd":3,"uuid":"f9403889-7fb8-4c49-8193-55ec6ae285cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":12,"last_clean_end":95,"up_from":148,"up_thru":233,"down_at":97,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6817","nonce":51640907}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6819","nonce":51640907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6823","nonce":51640907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":51640907},{"type":"v1","addr":"172.21.15.53:6821","nonce":51640907}]},"public_addr":"172.21.15.53:6817/51640907","cluster_addr":"172.21.15.53:6819/51640907","heartbeat_back_addr":"172.21.15.53:6823/51640907","heartbeat_front_addr":"172.21.15.53:6821/51640907","state":["exists","up"]},{"osd":4,"uuid":"cf4edbdc-6ec7-496f-94b6-7e6128956d63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6801","nonce":2074654185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6803","nonce":2074654185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6807","nonce":2074654185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":2074654185},{"type":"v1","addr":"172.21.15.53:6805","nonce":2074654185}]},"public_addr":"172.21.15.53:6801/2074654185","cluster_addr":"172.21.15.53:6803/2074654185","heartbeat_back_addr":"172.21.15.53:6807/2074654185","heartbeat_front_addr":"172.21.15.53:6805/2074654185","state":["exists","up"]},{"osd":5,"uuid":"ebca2188-456e-4d5c-956d-6382dd41b9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":233,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6809","nonce":2633163970}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6811","nonce":2633163970}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6815","nonce":2633163970}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":2633163970},{"type":"v1","addr":"172.21.15.53:6813","nonce":2633163970}]},"public_addr":"172.21.15.53:6809/2633163970","cluster_addr":"172.21.15.53:6811/2633163970","heartbeat_back_addr":"172.21.15.53:6815/2633163970","heartbeat_front_addr":"172.21.15.53:6813/2633163970","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.256552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.060071+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.852414+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.699046+0000","dead_epoch":97},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.762211+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T21:45:42.616713+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.12","osds":[1,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T21:50:30.881 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T21:50:30.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:31.022 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:31.023 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:31.023 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:31.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T21:50:31.320 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T21:50:31.333 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T21:50:31.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:31.458 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:31.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:31.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T21:50:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T21:50:31.782 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T21:50:31.783 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:31.911 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:31.911 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:31.911 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:31.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T21:50:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T21:50:32.225 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T21:50:32.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:32.351 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:32.351 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:32.351 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:32.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T21:50:32.653 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T21:50:32.666 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T21:50:32.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:32.797 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:32.798 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:32.798 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:32.809 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T21:50:33.099 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T21:50:33.113 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T21:50:33.113 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T21:50:33.238 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-17T21:50:33.238 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T21:50:33.239 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-17T21:50:33.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T21:50:33.541 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T21:50:33.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:33.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:33.856 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:33.868 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-17T21:50:32.149460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21480,"ondisk_log_size":21480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":360028,"kb_used_data":19584,"kb_used_omap":36,"kb_used_meta":340251,"kb_avail":565871012,"statfs":{"total":579820584960,"available":579451916288,"internally_reserved":0,"allocated":20054016,"data_stored":9161664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":348417321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.277883"},"pg_stats":[{"pgid":"3.1f","version":"247'418","reported_seq":11931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878390+0000","last_change":"2024-09-17T21:50:15.831227+0000","last_active":"2024-09-17T21:50:24.878390+0000","last_peered":"2024-09-17T21:50:24.878390+0000","last_clean":"2024-09-17T21:50:24.878390+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:24.878390+0000","last_undegraded":"2024-09-17T21:50:24.878390+0000","last_fullsized":"2024-09-17T21:50:24.878390+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":418,"log_dups_size":0,"ondisk_log_size":418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:33:00.335338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"246'611","reported_seq":20686,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879032+0000","last_change":"2024-09-17T21:50:09.807444+0000","last_active":"2024-09-17T21:50:24.879032+0000","last_peered":"2024-09-17T21:50:24.879032+0000","last_clean":"2024-09-17T21:50:24.879032+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:24.879032+0000","last_undegraded":"2024-09-17T21:50:24.879032+0000","last_fullsized":"2024-09-17T21:50:24.879032+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:54:20.858776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"247'416","reported_seq":5571,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851520+0000","last_change":"2024-09-17T21:50:09.824249+0000","last_active":"2024-09-17T21:50:25.851520+0000","last_peered":"2024-09-17T21:50:25.851520+0000","last_clean":"2024-09-17T21:50:25.851520+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:25.851520+0000","last_undegraded":"2024-09-17T21:50:25.851520+0000","last_fullsized":"2024-09-17T21:50:25.851520+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":416,"log_dups_size":0,"ondisk_log_size":416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:09:12.689643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"247'660","reported_seq":18673,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851786+0000","last_change":"2024-09-17T21:50:09.822738+0000","last_active":"2024-09-17T21:50:25.851786+0000","last_peered":"2024-09-17T21:50:25.851786+0000","last_clean":"2024-09-17T21:50:25.851786+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:25.851786+0000","last_undegraded":"2024-09-17T21:50:25.851786+0000","last_fullsized":"2024-09-17T21:50:25.851786+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:40:39.851439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"247'489","reported_seq":17159,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878388+0000","last_change":"2024-09-17T21:50:15.835058+0000","last_active":"2024-09-17T21:50:24.878388+0000","last_peered":"2024-09-17T21:50:24.878388+0000","last_clean":"2024-09-17T21:50:24.878388+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:24.878388+0000","last_undegraded":"2024-09-17T21:50:24.878388+0000","last_fullsized":"2024-09-17T21:50:24.878388+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":489,"log_dups_size":0,"ondisk_log_size":489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:22:28.108594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"246'731","reported_seq":12221,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851572+0000","last_change":"2024-09-17T21:50:09.813672+0000","last_active":"2024-09-17T21:50:25.851572+0000","last_peered":"2024-09-17T21:50:25.851572+0000","last_clean":"2024-09-17T21:50:25.851572+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:25.851572+0000","last_undegraded":"2024-09-17T21:50:25.851572+0000","last_fullsized":"2024-09-17T21:50:25.851572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:33.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"242'469","reported_seq":5261,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758479+0000","last_change":"2024-09-17T21:50:09.840726+0000","last_active":"2024-09-17T21:50:25.758479+0000","last_peered":"2024-09-17T21:50:25.758479+0000","last_clean":"2024-09-17T21:50:25.758479+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:25.758479+0000","last_undegraded":"2024-09-17T21:50:25.758479+0000","last_fullsized":"2024-09-17T21:50:25.758479+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:36.770626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"247'733","reported_seq":13469,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878777+0000","last_change":"2024-09-17T21:50:09.808306+0000","last_active":"2024-09-17T21:50:24.878777+0000","last_peered":"2024-09-17T21:50:24.878777+0000","last_clean":"2024-09-17T21:50:24.878777+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:24.878777+0000","last_undegraded":"2024-09-17T21:50:24.878777+0000","last_fullsized":"2024-09-17T21:50:24.878777+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:13:46.515052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"242'385","reported_seq":7841,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:15.833054+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:45:10.809759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"247'1025","reported_seq":12099,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851727+0000","last_change":"2024-09-17T21:50:09.823560+0000","last_active":"2024-09-17T21:50:25.851727+0000","last_peered":"2024-09-17T21:50:25.851727+0000","last_clean":"2024-09-17T21:50:25.851727+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:25.851727+0000","last_undegraded":"2024-09-17T21:50:25.851727+0000","last_fullsized":"2024-09-17T21:50:25.851727+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1025,"log_dups_size":0,"ondisk_log_size":1025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:09.809344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"242'790","reported_seq":10301,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878513+0000","last_change":"2024-09-17T21:50:09.809106+0000","last_active":"2024-09-17T21:50:24.878513+0000","last_peered":"2024-09-17T21:50:24.878513+0000","last_clean":"2024-09-17T21:50:24.878513+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:24.878513+0000","last_undegraded":"2024-09-17T21:50:24.878513+0000","last_fullsized":"2024-09-17T21:50:24.878513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:55:36.597584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"246'531","reported_seq":19436,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878357+0000","last_change":"2024-09-17T21:50:09.807989+0000","last_active":"2024-09-17T21:50:24.878357+0000","last_peered":"2024-09-17T21:50:24.878357+0000","last_clean":"2024-09-17T21:50:24.878357+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:24.878357+0000","last_undegraded":"2024-09-17T21:50:24.878357+0000","last_fullsized":"2024-09-17T21:50:24.878357+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":531,"log_dups_size":0,"ondisk_log_size":531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:54:59.506477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878632+0000","last_change":"2024-09-17T21:49:19.693249+0000","last_active":"2024-09-17T21:50:24.878632+0000","last_peered":"2024-09-17T21:50:24.878632+0000","last_clean":"2024-09-17T21:50:24.878632+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:24.878632+0000","last_undegraded":"2024-09-17T21:50:24.878632+0000","last_fullsized":"2024-09-17T21:50:24.878632+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:29:56.941033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"247'762","reported_seq":10900,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878328+0000","last_change":"2024-09-17T21:50:15.834176+0000","last_active":"2024-09-17T21:50:24.878328+0000","last_peered":"2024-09-17T21:50:24.878328+0000","last_clean":"2024-09-17T21:50:24.878328+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:24.878328+0000","last_undegraded":"2024-09-17T21:50:24.878328+0000","last_fullsized":"2024-09-17T21:50:24.878328+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:01:00.864891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":530,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758762+0000","last_change":"2024-09-17T21:49:19.693477+0000","last_active":"2024-09-17T21:50:25.758762+0000","last_peered":"2024-09-17T21:50:25.758762+0000","last_clean":"2024-09-17T21:50:25.758762+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:25.758762+0000","last_undegraded":"2024-09-17T21:50:25.758762+0000","last_fullsized":"2024-09-17T21:50:25.758762+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:41.626317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"246'569","reported_seq":6931,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.879064+0000","last_change":"2024-09-17T21:50:09.807442+0000","last_active":"2024-09-17T21:50:24.879064+0000","last_peered":"2024-09-17T21:50:24.879064+0000","last_clean":"2024-09-17T21:50:24.879064+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:24.879064+0000","last_undegraded":"2024-09-17T21:50:24.879064+0000","last_fullsized":"2024-09-17T21:50:24.879064+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":569,"log_dups_size":0,"ondisk_log_size":569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:54:02.835174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878614+0000","last_change":"2024-09-17T21:49:19.695032+0000","last_active":"2024-09-17T21:50:24.878614+0000","last_peered":"2024-09-17T21:50:24.878614+0000","last_clean":"2024-09-17T21:50:24.878614+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:24.878614+0000","last_undegraded":"2024-09-17T21:50:24.878614+0000","last_fullsized":"2024-09-17T21:50:24.878614+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:56:49.940884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"244'709","reported_seq":19556,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758682+0000","last_change":"2024-09-17T21:50:09.841290+0000","last_active":"2024-09-17T21:50:25.758682+0000","last_peered":"2024-09-17T21:50:25.758682+0000","last_clean":"2024-09-17T21:50:25.758682+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:25.758682+0000","last_undegraded":"2024-09-17T21:50:25.758682+0000","last_fullsized":"2024-09-17T21:50:25.758682+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":709,"log_dups_size":0,"ondisk_log_size":709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:36:20.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":538,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758792+0000","last_change":"2024-09-17T21:48:07.678293+0000","last_active":"2024-09-17T21:50:25.758792+0000","last_peered":"2024-09-17T21:50:25.758792+0000","last_clean":"2024-09-17T21:50:25.758792+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:25.758792+0000","last_undegraded":"2024-09-17T21:50:25.758792+0000","last_fullsized":"2024-09-17T21:50:25.758792+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:30:48.983676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"244'548","reported_seq":13127,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851660+0000","last_change":"2024-09-17T21:50:09.823316+0000","last_active":"2024-09-17T21:50:25.851660+0000","last_peered":"2024-09-17T21:50:25.851660+0000","last_clean":"2024-09-17T21:50:25.851660+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:25.851660+0000","last_undegraded":"2024-09-17T21:50:25.851660+0000","last_fullsized":"2024-09-17T21:50:25.851660+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:30:26.416654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"247'1073","reported_seq":12130,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851737+0000","last_change":"2024-09-17T21:50:09.814270+0000","last_active":"2024-09-17T21:50:25.851737+0000","last_peered":"2024-09-17T21:50:25.851737+0000","last_clean":"2024-09-17T21:50:25.851737+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:25.851737+0000","last_undegraded":"2024-09-17T21:50:25.851737+0000","last_fullsized":"2024-09-17T21:50:25.851737+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:15.620851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"19'2","reported_seq":504,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851383+0000","last_change":"2024-09-17T21:45:54.240753+0000","last_active":"2024-09-17T21:50:25.851383+0000","last_peered":"2024-09-17T21:50:25.851383+0000","last_clean":"2024-09-17T21:50:25.851383+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:25.851383+0000","last_undegraded":"2024-09-17T21:50:25.851383+0000","last_fullsized":"2024-09-17T21:50:25.851383+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:25:20.949909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"246'479","reported_seq":7534,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758358+0000","last_change":"2024-09-17T21:50:09.841344+0000","last_active":"2024-09-17T21:50:25.758358+0000","last_peered":"2024-09-17T21:50:25.758358+0000","last_clean":"2024-09-17T21:50:25.758358+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:25.758358+0000","last_undegraded":"2024-09-17T21:50:25.758358+0000","last_fullsized":"2024-09-17T21:50:25.758358+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:31:32.373044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":160,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878709+0000","last_change":"2024-09-17T21:49:07.908399+0000","last_active":"2024-09-17T21:50:24.878709+0000","last_peered":"2024-09-17T21:50:24.878709+0000","last_clean":"2024-09-17T21:50:24.878709+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:24.878709+0000","last_undegraded":"2024-09-17T21:50:24.878709+0000","last_fullsized":"2024-09-17T21:50:24.878709+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:48:33.713074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"247'1934","reported_seq":9361,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878179+0000","last_change":"2024-09-17T21:50:15.833959+0000","last_active":"2024-09-17T21:50:24.878179+0000","last_peered":"2024-09-17T21:50:24.878179+0000","last_clean":"2024-09-17T21:50:24.878179+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:24.878179+0000","last_undegraded":"2024-09-17T21:50:24.878179+0000","last_fullsized":"2024-09-17T21:50:24.878179+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1934,"log_dups_size":0,"ondisk_log_size":1934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:46:34.122775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":136,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878269+0000","last_change":"2024-09-17T21:49:19.694024+0000","last_active":"2024-09-17T21:50:24.878269+0000","last_peered":"2024-09-17T21:50:24.878269+0000","last_clean":"2024-09-17T21:50:24.878269+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:24.878269+0000","last_undegraded":"2024-09-17T21:50:24.878269+0000","last_fullsized":"2024-09-17T21:50:24.878269+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:15:28.922373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"246'525","reported_seq":5855,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758505+0000","last_change":"2024-09-17T21:50:09.842770+0000","last_active":"2024-09-17T21:50:25.758505+0000","last_peered":"2024-09-17T21:50:25.758505+0000","last_clean":"2024-09-17T21:50:25.758505+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:25.758505+0000","last_undegraded":"2024-09-17T21:50:25.758505+0000","last_fullsized":"2024-09-17T21:50:25.758505+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:06:10.667781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"17'1","reported_seq":521,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851484+0000","last_change":"2024-09-17T21:49:07.912554+0000","last_active":"2024-09-17T21:50:25.851484+0000","last_peered":"2024-09-17T21:50:25.851484+0000","last_clean":"2024-09-17T21:50:25.851484+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:25.851484+0000","last_undegraded":"2024-09-17T21:50:25.851484+0000","last_fullsized":"2024-09-17T21:50:25.851484+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:51.221520+0000","last_clean_scrub_stamp":"2024-09-17T21:45:51.221520+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:40:00.226784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":650,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878278+0000","last_change":"2024-09-17T21:49:22.711261+0000","last_active":"2024-09-17T21:50:24.878278+0000","last_peered":"2024-09-17T21:50:24.878278+0000","last_clean":"2024-09-17T21:50:24.878278+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:24.878278+0000","last_undegraded":"2024-09-17T21:50:24.878278+0000","last_fullsized":"2024-09-17T21:50:24.878278+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:45:46.146831+0000","last_clean_scrub_stamp":"2024-09-17T21:45:46.146831+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:06:38.327973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"246'492","reported_seq":7478,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878906+0000","last_change":"2024-09-17T21:50:09.808146+0000","last_active":"2024-09-17T21:50:24.878906+0000","last_peered":"2024-09-17T21:50:24.878906+0000","last_clean":"2024-09-17T21:50:24.878906+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:24.878906+0000","last_undegraded":"2024-09-17T21:50:24.878906+0000","last_fullsized":"2024-09-17T21:50:24.878906+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":492,"log_dups_size":0,"ondisk_log_size":492,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:52:29.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"246'852","reported_seq":11202,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851654+0000","last_change":"2024-09-17T21:50:09.814175+0000","last_active":"2024-09-17T21:50:25.851654+0000","last_peered":"2024-09-17T21:50:25.851654+0000","last_clean":"2024-09-17T21:50:25.851654+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:25.851654+0000","last_undegraded":"2024-09-17T21:50:25.851654+0000","last_fullsized":"2024-09-17T21:50:25.851654+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:04.365208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"246'548","reported_seq":5467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878367+0000","last_change":"2024-09-17T21:50:09.807996+0000","last_active":"2024-09-17T21:50:24.878367+0000","last_peered":"2024-09-17T21:50:24.878367+0000","last_clean":"2024-09-17T21:50:24.878367+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:24.878367+0000","last_undegraded":"2024-09-17T21:50:24.878367+0000","last_fullsized":"2024-09-17T21:50:24.878367+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:40.644277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"242'589","reported_seq":11095,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758555+0000","last_change":"2024-09-17T21:50:09.840660+0000","last_active":"2024-09-17T21:50:25.758555+0000","last_peered":"2024-09-17T21:50:25.758555+0000","last_clean":"2024-09-17T21:50:25.758555+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:25.758555+0000","last_undegraded":"2024-09-17T21:50:25.758555+0000","last_fullsized":"2024-09-17T21:50:25.758555+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:22:28.483964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"247'621","reported_seq":19414,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851587+0000","last_change":"2024-09-17T21:50:09.824971+0000","last_active":"2024-09-17T21:50:25.851587+0000","last_peered":"2024-09-17T21:50:25.851587+0000","last_clean":"2024-09-17T21:50:25.851587+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:25.851587+0000","last_undegraded":"2024-09-17T21:50:25.851587+0000","last_fullsized":"2024-09-17T21:50:25.851587+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:36:44.186239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"245'886","reported_seq":10280,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878615+0000","last_change":"2024-09-17T21:50:09.810044+0000","last_active":"2024-09-17T21:50:24.878615+0000","last_peered":"2024-09-17T21:50:24.878615+0000","last_clean":"2024-09-17T21:50:24.878615+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:24.878615+0000","last_undegraded":"2024-09-17T21:50:24.878615+0000","last_fullsized":"2024-09-17T21:50:24.878615+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:00:23.122701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"247'437","reported_seq":6163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851682+0000","last_change":"2024-09-17T21:50:09.824972+0000","last_active":"2024-09-17T21:50:25.851682+0000","last_peered":"2024-09-17T21:50:25.851682+0000","last_clean":"2024-09-17T21:50:25.851682+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:25.851682+0000","last_undegraded":"2024-09-17T21:50:25.851682+0000","last_fullsized":"2024-09-17T21:50:25.851682+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:04.630383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"244'1029","reported_seq":17757,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851728+0000","last_change":"2024-09-17T21:50:09.824388+0000","last_active":"2024-09-17T21:50:25.851728+0000","last_peered":"2024-09-17T21:50:25.851728+0000","last_clean":"2024-09-17T21:50:25.851728+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:25.851728+0000","last_undegraded":"2024-09-17T21:50:25.851728+0000","last_fullsized":"2024-09-17T21:50:25.851728+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":1029,"log_dups_size":0,"ondisk_log_size":1029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:30:53.294173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"244'623","reported_seq":7463,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.851349+0000","last_change":"2024-09-17T21:50:09.822710+0000","last_active":"2024-09-17T21:50:25.851349+0000","last_peered":"2024-09-17T21:50:25.851349+0000","last_clean":"2024-09-17T21:50:25.851349+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:25.851349+0000","last_undegraded":"2024-09-17T21:50:25.851349+0000","last_fullsized":"2024-09-17T21:50:25.851349+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:56:00.042872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"246'554","reported_seq":6454,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878270+0000","last_change":"2024-09-17T21:50:09.806855+0000","last_active":"2024-09-17T21:50:24.878270+0000","last_peered":"2024-09-17T21:50:24.878270+0000","last_clean":"2024-09-17T21:50:24.878270+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:24.878270+0000","last_undegraded":"2024-09-17T21:50:24.878270+0000","last_fullsized":"2024-09-17T21:50:24.878270+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:00:42.269040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"245'390","reported_seq":12760,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:25.758285+0000","last_change":"2024-09-17T21:50:09.842163+0000","last_active":"2024-09-17T21:50:25.758285+0000","last_peered":"2024-09-17T21:50:25.758285+0000","last_clean":"2024-09-17T21:50:25.758285+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:25.758285+0000","last_undegraded":"2024-09-17T21:50:25.758285+0000","last_fullsized":"2024-09-17T21:50:25.758285+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:34:43.065656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"247'482","reported_seq":10467,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:24.878326+0000","last_change":"2024-09-17T21:50:09.806881+0000","last_active":"2024-09-17T21:50:24.878326+0000","last_peered":"2024-09-17T21:50:24.878326+0000","last_clean":"2024-09-17T21:50:24.878326+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:24.878326+0000","last_undegraded":"2024-09-17T21:50:24.878326+0000","last_fullsized":"2024-09-17T21:50:24.878326+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:46:00.249865+0000","last_clean_scrub_stamp":"2024-09-17T21:46:00.249865+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:18:56.089161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":21360,"ondisk_log_size":21360,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607614,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60636,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311204,"statfs":{"total":96636764160,"available":96574672896,"internally_reserved":0,"allocated":3145728,"data_stored":1330161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159834,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63196,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308644,"statfs":{"total":96636764160,"available":96572051456,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607613,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56468,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315372,"statfs":{"total":96636764160,"available":96578940928,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607613,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48732,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323108,"statfs":{"total":96636764160,"available":96586862592,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607613,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70364,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301476,"statfs":{"total":96636764160,"available":96564711424,"internally_reserved":0,"allocated":3145728,"data_stored":1330161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640317,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60632,"kb_used_data":3076,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311208,"statfs":{"total":96636764160,"available":96574676992,"internally_reserved":0,"allocated":3149824,"data_stored":1330180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:33.873 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.873 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.873 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.874 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.875 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-17T21:45:51.221520+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=51, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T21:45:46.146831+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=45, tm_sec=46, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.876 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.877 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.878 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.878 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-17T21:46:00.249865+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=46, tm_sec=0, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=21, tm_min=50, tm_sec=29, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T21:50:33.878 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T21:50:53.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T21:50:54.183 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-17T21:50:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-17T21:50:54.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-17T21:50:54.154477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":716,"num_object_clones":0,"num_object_copies":1432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":716,"num_whiteouts":708,"num_read":382189,"num_read_kb":1876993,"num_write":23071,"num_write_kb":604779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11222,"num_bytes_recovered":1262586915,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22193,"ondisk_log_size":22193,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":364128,"kb_used_data":19584,"kb_used_omap":36,"kb_used_meta":344411,"kb_avail":565866912,"statfs":{"total":579820584960,"available":579447717888,"internally_reserved":0,"allocated":20054016,"data_stored":9161664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":37591,"internal_metadata":352677161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002831"},"pg_stats":[{"pgid":"3.1f","version":"260'443","reported_seq":11966,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:31.744767+0000","last_change":"2024-09-17T21:50:31.744790+0000","last_active":"2024-09-17T21:50:31.744767+0000","last_peered":"2024-09-17T21:50:31.744767+0000","last_clean":"2024-09-17T21:50:31.744767+0000","last_became_active":"2024-09-17T21:50:15.829732+0000","last_became_peered":"2024-09-17T21:50:15.829732+0000","last_unstale":"2024-09-17T21:50:31.744767+0000","last_undegraded":"2024-09-17T21:50:31.744767+0000","last_fullsized":"2024-09-17T21:50:31.744767+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'443","last_scrub_stamp":"2024-09-17T21:50:31.744678+0000","last_deep_scrub":"260'443","last_deep_scrub_stamp":"2024-09-17T21:50:31.744678+0000","last_clean_scrub_stamp":"2024-09-17T21:50:31.744678+0000","objects_scrubbed":25,"log_size":443,"log_dups_size":0,"ondisk_log_size":443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:26:06.260703+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0.0037858079999999999,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":11478,"num_read_kb":71061,"num_write":379,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":787,"num_bytes_recovered":68995721,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"260'635","reported_seq":20719,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:36.812885+0000","last_change":"2024-09-17T21:50:36.812949+0000","last_active":"2024-09-17T21:50:36.812885+0000","last_peered":"2024-09-17T21:50:36.812885+0000","last_clean":"2024-09-17T21:50:36.812885+0000","last_became_active":"2024-09-17T21:49:23.715274+0000","last_became_peered":"2024-09-17T21:49:23.715274+0000","last_unstale":"2024-09-17T21:50:36.812885+0000","last_undegraded":"2024-09-17T21:50:36.812885+0000","last_fullsized":"2024-09-17T21:50:36.812885+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'635","last_scrub_stamp":"2024-09-17T21:50:36.812802+0000","last_deep_scrub":"260'635","last_deep_scrub_stamp":"2024-09-17T21:50:36.812802+0000","last_clean_scrub_stamp":"2024-09-17T21:50:36.812802+0000","objects_scrubbed":24,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:29:27.607230+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00152407,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":19155,"num_read_kb":109404,"num_write":455,"num_write_kb":34499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":837,"num_bytes_recovered":160812784,"num_keys_recovered":144,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"260'441","reported_seq":5606,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.851787+0000","last_change":"2024-09-17T21:50:35.851816+0000","last_active":"2024-09-17T21:50:35.851787+0000","last_peered":"2024-09-17T21:50:35.851787+0000","last_clean":"2024-09-17T21:50:35.851787+0000","last_became_active":"2024-09-17T21:47:10.889068+0000","last_became_peered":"2024-09-17T21:47:10.889068+0000","last_unstale":"2024-09-17T21:50:35.851787+0000","last_undegraded":"2024-09-17T21:50:35.851787+0000","last_fullsized":"2024-09-17T21:50:35.851787+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'441","last_scrub_stamp":"2024-09-17T21:50:35.851701+0000","last_deep_scrub":"260'441","last_deep_scrub_stamp":"2024-09-17T21:50:35.851701+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.851701+0000","objects_scrubbed":25,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:22:45.098310+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.002307262,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4928,"num_read_kb":51145,"num_write":666,"num_write_kb":31823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":10803273,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"260'680","reported_seq":18702,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:36.888662+0000","last_change":"2024-09-17T21:50:36.888744+0000","last_active":"2024-09-17T21:50:36.888662+0000","last_peered":"2024-09-17T21:50:36.888662+0000","last_clean":"2024-09-17T21:50:36.888662+0000","last_became_active":"2024-09-17T21:47:11.889767+0000","last_became_peered":"2024-09-17T21:47:11.889767+0000","last_unstale":"2024-09-17T21:50:36.888662+0000","last_undegraded":"2024-09-17T21:50:36.888662+0000","last_fullsized":"2024-09-17T21:50:36.888662+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'680","last_scrub_stamp":"2024-09-17T21:50:36.888573+0000","last_deep_scrub":"260'680","last_deep_scrub_stamp":"2024-09-17T21:50:36.888573+0000","last_clean_scrub_stamp":"2024-09-17T21:50:36.888573+0000","objects_scrubbed":21,"log_size":680,"log_dups_size":0,"ondisk_log_size":680,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:06:01.467636+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.000953918,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":20,"num_read":19906,"num_read_kb":87073,"num_write":721,"num_write_kb":39754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":92,"num_bytes_recovered":54487642,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"260'505","reported_seq":17184,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.763323+0000","last_change":"2024-09-17T21:50:35.763368+0000","last_active":"2024-09-17T21:50:35.763323+0000","last_peered":"2024-09-17T21:50:35.763323+0000","last_clean":"2024-09-17T21:50:35.763323+0000","last_became_active":"2024-09-17T21:50:15.832180+0000","last_became_peered":"2024-09-17T21:50:15.832180+0000","last_unstale":"2024-09-17T21:50:35.763323+0000","last_undegraded":"2024-09-17T21:50:35.763323+0000","last_fullsized":"2024-09-17T21:50:35.763323+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'505","last_scrub_stamp":"2024-09-17T21:50:35.763231+0000","last_deep_scrub":"260'505","last_deep_scrub_stamp":"2024-09-17T21:50:35.763231+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.763231+0000","objects_scrubbed":16,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:08:53.762569+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014332629999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":16,"num_read":14689,"num_read_kb":43738,"num_write":383,"num_write_kb":8472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":18718224,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"260'755","reported_seq":12254,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.316799+0000","last_change":"2024-09-17T21:50:33.316851+0000","last_active":"2024-09-17T21:50:33.316799+0000","last_peered":"2024-09-17T21:50:33.316799+0000","last_clean":"2024-09-17T21:50:33.316799+0000","last_became_active":"2024-09-17T21:47:10.888524+0000","last_became_peered":"2024-09-17T21:47:10.888524+0000","last_unstale":"2024-09-17T21:50:33.316799+0000","last_undegraded":"2024-09-17T21:50:33.316799+0000","last_fullsized":"2024-09-17T21:50:33.316799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'755","last_scrub_stamp":"2024-09-17T21:50:33.316709+0000","last_deep_scrub":"260'755","last_deep_scrub_stamp":"2024-09-17T21:50:33.316709+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.316709+0000","objects_scrubbed":24,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:32:40.691433+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.0013446070000000001,"stat_sum":{"num_bytes":19,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":23,"num_read":14136,"num_read_kb":82655,"num_write":739,"num_write_kb":25562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":32634612,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"260'494","reported_seq":5297,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.793009+0000","last_change":"2024-09-17T21:50:33.793009+0000","last_active":"2024-09-17T21:50:33.793009+0000","last_peered":"2024-09-17T21:50:33.793009+0000","last_clean":"2024-09-17T21:50:33.793009+0000","last_became_active":"2024-09-17T21:47:11.889514+0000","last_became_peered":"2024-09-17T21:47:11.889514+0000","last_unstale":"2024-09-17T21:50:33.793009+0000","last_undegraded":"2024-09-17T21:50:33.793009+0000","last_fullsized":"2024-09-17T21:50:33.793009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'494","last_scrub_stamp":"2024-09-17T21:50:33.792904+0000","last_deep_scrub":"260'494","last_deep_scrub_stamp":"2024-09-17T21:50:33.792904+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.792904+0000","objects_scrubbed":25,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:21:31.990912+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00092048599999999998,"stat_sum":{"num_bytes":0,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":25,"num_read":4665,"num_read_kb":34117,"num_write":461,"num_write_kb":13460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":11000384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"260'755","reported_seq":13500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.768482+0000","last_change":"2024-09-17T21:50:33.768561+0000","last_active":"2024-09-17T21:50:33.768482+0000","last_peered":"2024-09-17T21:50:33.768482+0000","last_clean":"2024-09-17T21:50:33.768482+0000","last_became_active":"2024-09-17T21:49:10.914278+0000","last_became_peered":"2024-09-17T21:49:10.914278+0000","last_unstale":"2024-09-17T21:50:33.768482+0000","last_undegraded":"2024-09-17T21:50:33.768482+0000","last_fullsized":"2024-09-17T21:50:33.768482+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'755","last_scrub_stamp":"2024-09-17T21:50:33.768395+0000","last_deep_scrub":"260'755","last_deep_scrub_stamp":"2024-09-17T21:50:33.768395+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.768395+0000","objects_scrubbed":22,"log_size":755,"log_dups_size":0,"ondisk_log_size":755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:05:26.068921+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.001767841,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":12590,"num_read_kb":69211,"num_write":826,"num_write_kb":13815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":15629390,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"260'411","reported_seq":7878,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:32.740028+0000","last_change":"2024-09-17T21:50:32.740028+0000","last_active":"2024-09-17T21:50:32.740028+0000","last_peered":"2024-09-17T21:50:32.740028+0000","last_clean":"2024-09-17T21:50:32.740028+0000","last_became_active":"2024-09-17T21:50:15.831721+0000","last_became_peered":"2024-09-17T21:50:15.831721+0000","last_unstale":"2024-09-17T21:50:32.740028+0000","last_undegraded":"2024-09-17T21:50:32.740028+0000","last_fullsized":"2024-09-17T21:50:32.740028+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'411","last_scrub_stamp":"2024-09-17T21:50:32.739852+0000","last_deep_scrub":"260'411","last_deep_scrub_stamp":"2024-09-17T21:50:32.739852+0000","last_clean_scrub_stamp":"2024-09-17T21:50:32.739852+0000","objects_scrubbed":26,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:47:08.335764+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001894338,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":7903,"num_read_kb":32043,"num_write":349,"num_write_kb":10117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":13178368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"260'1058","reported_seq":12143,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:42.849773+0000","last_change":"2024-09-17T21:50:42.849773+0000","last_active":"2024-09-17T21:50:42.849773+0000","last_peered":"2024-09-17T21:50:42.849773+0000","last_clean":"2024-09-17T21:50:42.849773+0000","last_became_active":"2024-09-17T21:49:23.715286+0000","last_became_peered":"2024-09-17T21:49:23.715286+0000","last_unstale":"2024-09-17T21:50:42.849773+0000","last_undegraded":"2024-09-17T21:50:42.849773+0000","last_fullsized":"2024-09-17T21:50:42.849773+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1058","last_scrub_stamp":"2024-09-17T21:50:42.849650+0000","last_deep_scrub":"260'1058","last_deep_scrub_stamp":"2024-09-17T21:50:42.849650+0000","last_clean_scrub_stamp":"2024-09-17T21:50:42.849650+0000","objects_scrubbed":33,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:12.781538+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0.001700294,"stat_sum":{"num_bytes":0,"num_objects":33,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":33,"num_read":14402,"num_read_kb":74351,"num_write":1045,"num_write_kb":17171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":584,"num_bytes_recovered":59984709,"num_keys_recovered":188,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"260'812","reported_seq":10332,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:40.398010+0000","last_change":"2024-09-17T21:50:40.398074+0000","last_active":"2024-09-17T21:50:40.398010+0000","last_peered":"2024-09-17T21:50:40.398010+0000","last_clean":"2024-09-17T21:50:40.398010+0000","last_became_active":"2024-09-17T21:49:23.714346+0000","last_became_peered":"2024-09-17T21:49:23.714346+0000","last_unstale":"2024-09-17T21:50:40.398010+0000","last_undegraded":"2024-09-17T21:50:40.398010+0000","last_fullsized":"2024-09-17T21:50:40.398010+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'812","last_scrub_stamp":"2024-09-17T21:50:40.397910+0000","last_deep_scrub":"260'812","last_deep_scrub_stamp":"2024-09-17T21:50:40.397910+0000","last_clean_scrub_stamp":"2024-09-17T21:50:40.397910+0000","objects_scrubbed":22,"log_size":812,"log_dups_size":0,"ondisk_log_size":812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:11:43.800749+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.003333379,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":17180,"num_read_kb":60033,"num_write":736,"num_write_kb":14205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":40867582,"num_keys_recovered":175,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"260'548","reported_seq":19462,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:42.381996+0000","last_change":"2024-09-17T21:50:42.382053+0000","last_active":"2024-09-17T21:50:42.381996+0000","last_peered":"2024-09-17T21:50:42.381996+0000","last_clean":"2024-09-17T21:50:42.381996+0000","last_became_active":"2024-09-17T21:49:33.673059+0000","last_became_peered":"2024-09-17T21:49:33.673059+0000","last_unstale":"2024-09-17T21:50:42.381996+0000","last_undegraded":"2024-09-17T21:50:42.381996+0000","last_fullsized":"2024-09-17T21:50:42.381996+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'548","last_scrub_stamp":"2024-09-17T21:50:42.381892+0000","last_deep_scrub":"260'548","last_deep_scrub_stamp":"2024-09-17T21:50:42.381892+0000","last_clean_scrub_stamp":"2024-09-17T21:50:42.381892+0000","objects_scrubbed":17,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:15:35.211575+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0023832889999999998,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":18246,"num_read_kb":143652,"num_write":571,"num_write_kb":44005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":54119764,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.448081+0000","last_change":"2024-09-17T21:50:33.448105+0000","last_active":"2024-09-17T21:50:33.448081+0000","last_peered":"2024-09-17T21:50:33.448081+0000","last_clean":"2024-09-17T21:50:33.448081+0000","last_became_active":"2024-09-17T21:49:19.692890+0000","last_became_peered":"2024-09-17T21:49:19.692890+0000","last_unstale":"2024-09-17T21:50:33.448081+0000","last_undegraded":"2024-09-17T21:50:33.448081+0000","last_fullsized":"2024-09-17T21:50:33.448081+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:33.447969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:33.447969+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.447969+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T01:51:53.810532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044036599999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"260'785","reported_seq":10932,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.775247+0000","last_change":"2024-09-17T21:50:34.775315+0000","last_active":"2024-09-17T21:50:34.775247+0000","last_peered":"2024-09-17T21:50:34.775247+0000","last_clean":"2024-09-17T21:50:34.775247+0000","last_became_active":"2024-09-17T21:50:15.831894+0000","last_became_peered":"2024-09-17T21:50:15.831894+0000","last_unstale":"2024-09-17T21:50:34.775247+0000","last_undegraded":"2024-09-17T21:50:34.775247+0000","last_fullsized":"2024-09-17T21:50:34.775247+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'785","last_scrub_stamp":"2024-09-17T21:50:34.775164+0000","last_deep_scrub":"260'785","last_deep_scrub_stamp":"2024-09-17T21:50:34.775164+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.775164+0000","objects_scrubbed":23,"log_size":785,"log_dups_size":0,"ondisk_log_size":785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:59:08.547233+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.0021597159999999999,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":12164,"num_read_kb":53862,"num_write":616,"num_write_kb":16802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":55722435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":539,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:38.792674+0000","last_change":"2024-09-17T21:50:38.792744+0000","last_active":"2024-09-17T21:50:38.792674+0000","last_peered":"2024-09-17T21:50:38.792674+0000","last_clean":"2024-09-17T21:50:38.792674+0000","last_became_active":"2024-09-17T21:49:19.693154+0000","last_became_peered":"2024-09-17T21:49:19.693154+0000","last_unstale":"2024-09-17T21:50:38.792674+0000","last_undegraded":"2024-09-17T21:50:38.792674+0000","last_fullsized":"2024-09-17T21:50:38.792674+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:38.792540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:38.792540+0000","last_clean_scrub_stamp":"2024-09-17T21:50:38.792540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:40:53.293310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057388000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"260'591","reported_seq":6962,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.772628+0000","last_change":"2024-09-17T21:50:34.772733+0000","last_active":"2024-09-17T21:50:34.772628+0000","last_peered":"2024-09-17T21:50:34.772628+0000","last_clean":"2024-09-17T21:50:34.772628+0000","last_became_active":"2024-09-17T21:49:11.915057+0000","last_became_peered":"2024-09-17T21:49:11.915057+0000","last_unstale":"2024-09-17T21:50:34.772628+0000","last_undegraded":"2024-09-17T21:50:34.772628+0000","last_fullsized":"2024-09-17T21:50:34.772628+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'591","last_scrub_stamp":"2024-09-17T21:50:34.772516+0000","last_deep_scrub":"260'591","last_deep_scrub_stamp":"2024-09-17T21:50:34.772516+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.772516+0000","objects_scrubbed":22,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:10:37.016056+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.0012951779999999999,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":6415,"num_read_kb":43107,"num_write":642,"num_write_kb":14140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":14120786,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.360185+0000","last_change":"2024-09-17T21:50:35.360211+0000","last_active":"2024-09-17T21:50:35.360185+0000","last_peered":"2024-09-17T21:50:35.360185+0000","last_clean":"2024-09-17T21:50:35.360185+0000","last_became_active":"2024-09-17T21:49:19.694714+0000","last_became_peered":"2024-09-17T21:49:19.694714+0000","last_unstale":"2024-09-17T21:50:35.360185+0000","last_undegraded":"2024-09-17T21:50:35.360185+0000","last_fullsized":"2024-09-17T21:50:35.360185+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:35.360087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:35.360087+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.360087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:44:17.561703+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00079031100000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"260'729","reported_seq":19585,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:36.807753+0000","last_change":"2024-09-17T21:50:36.807813+0000","last_active":"2024-09-17T21:50:36.807753+0000","last_peered":"2024-09-17T21:50:36.807753+0000","last_clean":"2024-09-17T21:50:36.807753+0000","last_became_active":"2024-09-17T21:49:11.915327+0000","last_became_peered":"2024-09-17T21:49:11.915327+0000","last_unstale":"2024-09-17T21:50:36.807753+0000","last_undegraded":"2024-09-17T21:50:36.807753+0000","last_fullsized":"2024-09-17T21:50:36.807753+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'729","last_scrub_stamp":"2024-09-17T21:50:36.807653+0000","last_deep_scrub":"260'729","last_deep_scrub_stamp":"2024-09-17T21:50:36.807653+0000","last_clean_scrub_stamp":"2024-09-17T21:50:36.807653+0000","objects_scrubbed":20,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:01:44.888008+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0014184340000000001,"stat_sum":{"num_bytes":0,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":20,"num_read":19036,"num_read_kb":88547,"num_write":509,"num_write_kb":12143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":45187744,"num_keys_recovered":223,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":547,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.800065+0000","last_change":"2024-09-17T21:50:34.800088+0000","last_active":"2024-09-17T21:50:34.800065+0000","last_peered":"2024-09-17T21:50:34.800065+0000","last_clean":"2024-09-17T21:50:34.800065+0000","last_became_active":"2024-09-17T21:48:07.677980+0000","last_became_peered":"2024-09-17T21:48:07.677980+0000","last_unstale":"2024-09-17T21:50:34.800065+0000","last_undegraded":"2024-09-17T21:50:34.800065+0000","last_fullsized":"2024-09-17T21:50:34.800065+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:34.799983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:34.799983+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.799983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:32:17.541155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000575648,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"260'574","reported_seq":13163,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:40.837359+0000","last_change":"2024-09-17T21:50:40.837392+0000","last_active":"2024-09-17T21:50:40.837359+0000","last_peered":"2024-09-17T21:50:40.837359+0000","last_clean":"2024-09-17T21:50:40.837359+0000","last_became_active":"2024-09-17T21:49:11.914576+0000","last_became_peered":"2024-09-17T21:49:11.914576+0000","last_unstale":"2024-09-17T21:50:40.837359+0000","last_undegraded":"2024-09-17T21:50:40.837359+0000","last_fullsized":"2024-09-17T21:50:40.837359+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'574","last_scrub_stamp":"2024-09-17T21:50:40.837270+0000","last_deep_scrub":"260'574","last_deep_scrub_stamp":"2024-09-17T21:50:40.837270+0000","last_clean_scrub_stamp":"2024-09-17T21:50:40.837270+0000","objects_scrubbed":26,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:23:20.155427+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.001522745,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":11616,"num_read_kb":39661,"num_write":531,"num_write_kb":17816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":27736032,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"260'1088","reported_seq":12154,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.307252+0000","last_change":"2024-09-17T21:50:34.307333+0000","last_active":"2024-09-17T21:50:34.307252+0000","last_peered":"2024-09-17T21:50:34.307252+0000","last_clean":"2024-09-17T21:50:34.307252+0000","last_became_active":"2024-09-17T21:48:07.677884+0000","last_became_peered":"2024-09-17T21:48:07.677884+0000","last_unstale":"2024-09-17T21:50:34.307252+0000","last_undegraded":"2024-09-17T21:50:34.307252+0000","last_fullsized":"2024-09-17T21:50:34.307252+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1088","last_scrub_stamp":"2024-09-17T21:50:34.307135+0000","last_deep_scrub":"260'1088","last_deep_scrub_stamp":"2024-09-17T21:50:34.307135+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.307135+0000","objects_scrubbed":16,"log_size":1088,"log_dups_size":0,"ondisk_log_size":1088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:20:41.376243+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0019294449999999999,"stat_sum":{"num_bytes":0,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":15,"num_read":14491,"num_read_kb":39009,"num_write":1295,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":5142577,"num_keys_recovered":84,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"260'3","reported_seq":514,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.843552+0000","last_change":"2024-09-17T21:50:33.843580+0000","last_active":"2024-09-17T21:50:33.843552+0000","last_peered":"2024-09-17T21:50:33.843552+0000","last_clean":"2024-09-17T21:50:33.843552+0000","last_became_active":"2024-09-17T21:45:52.236645+0000","last_became_peered":"2024-09-17T21:45:52.236645+0000","last_unstale":"2024-09-17T21:50:33.843552+0000","last_undegraded":"2024-09-17T21:50:33.843552+0000","last_fullsized":"2024-09-17T21:50:33.843552+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'3","last_scrub_stamp":"2024-09-17T21:50:33.843467+0000","last_deep_scrub":"260'3","last_deep_scrub_stamp":"2024-09-17T21:50:33.843467+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.843467+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T02:03:38.541477+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00046965700000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"260'502","reported_seq":7566,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:37.811846+0000","last_change":"2024-09-17T21:50:37.811918+0000","last_active":"2024-09-17T21:50:37.811846+0000","last_peered":"2024-09-17T21:50:37.811846+0000","last_clean":"2024-09-17T21:50:37.811846+0000","last_became_active":"2024-09-17T21:49:11.915652+0000","last_became_peered":"2024-09-17T21:49:11.915652+0000","last_unstale":"2024-09-17T21:50:37.811846+0000","last_undegraded":"2024-09-17T21:50:37.811846+0000","last_fullsized":"2024-09-17T21:50:37.811846+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'502","last_scrub_stamp":"2024-09-17T21:50:37.811749+0000","last_deep_scrub":"260'502","last_deep_scrub_stamp":"2024-09-17T21:50:37.811749+0000","last_clean_scrub_stamp":"2024-09-17T21:50:37.811749+0000","objects_scrubbed":23,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:31:09.443002+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0.001420953,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10322,"num_read_kb":30449,"num_write":377,"num_write_kb":6768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":4126966,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":169,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:32.755777+0000","last_change":"2024-09-17T21:50:32.755804+0000","last_active":"2024-09-17T21:50:32.755777+0000","last_peered":"2024-09-17T21:50:32.755777+0000","last_clean":"2024-09-17T21:50:32.755777+0000","last_became_active":"2024-09-17T21:49:07.908073+0000","last_became_peered":"2024-09-17T21:49:07.908073+0000","last_unstale":"2024-09-17T21:50:32.755777+0000","last_undegraded":"2024-09-17T21:50:32.755777+0000","last_fullsized":"2024-09-17T21:50:32.755777+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:32.755684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:32.755684+0000","last_clean_scrub_stamp":"2024-09-17T21:50:32.755684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:13:01.683146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"260'1960","reported_seq":9397,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:33.736680+0000","last_change":"2024-09-17T21:50:33.736716+0000","last_active":"2024-09-17T21:50:33.736680+0000","last_peered":"2024-09-17T21:50:33.736680+0000","last_clean":"2024-09-17T21:50:33.736680+0000","last_became_active":"2024-09-17T21:50:15.831679+0000","last_became_peered":"2024-09-17T21:50:15.831679+0000","last_unstale":"2024-09-17T21:50:33.736680+0000","last_undegraded":"2024-09-17T21:50:33.736680+0000","last_fullsized":"2024-09-17T21:50:33.736680+0000","mapping_epoch":250,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":251,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1960","last_scrub_stamp":"2024-09-17T21:50:33.736610+0000","last_deep_scrub":"260'1960","last_deep_scrub_stamp":"2024-09-17T21:50:33.736610+0000","last_clean_scrub_stamp":"2024-09-17T21:50:33.736610+0000","objects_scrubbed":26,"log_size":1960,"log_dups_size":0,"ondisk_log_size":1960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:49:29.567954+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00090984400000000004,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":15898,"num_read_kb":57114,"num_write":2808,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":331,"num_bytes_recovered":20451125,"num_keys_recovered":177,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":145,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.408189+0000","last_change":"2024-09-17T21:50:34.408220+0000","last_active":"2024-09-17T21:50:34.408189+0000","last_peered":"2024-09-17T21:50:34.408189+0000","last_clean":"2024-09-17T21:50:34.408189+0000","last_became_active":"2024-09-17T21:49:19.693693+0000","last_became_peered":"2024-09-17T21:49:19.693693+0000","last_unstale":"2024-09-17T21:50:34.408189+0000","last_undegraded":"2024-09-17T21:50:34.408189+0000","last_fullsized":"2024-09-17T21:50:34.408189+0000","mapping_epoch":196,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":197,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T21:50:34.408077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T21:50:34.408077+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.408077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T00:59:55.291139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"260'548","reported_seq":5887,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:32.744289+0000","last_change":"2024-09-17T21:50:32.744355+0000","last_active":"2024-09-17T21:50:32.744289+0000","last_peered":"2024-09-17T21:50:32.744289+0000","last_clean":"2024-09-17T21:50:32.744289+0000","last_became_active":"2024-09-17T21:46:01.268189+0000","last_became_peered":"2024-09-17T21:46:01.268189+0000","last_unstale":"2024-09-17T21:50:32.744289+0000","last_undegraded":"2024-09-17T21:50:32.744289+0000","last_fullsized":"2024-09-17T21:50:32.744289+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'548","last_scrub_stamp":"2024-09-17T21:50:32.744209+0000","last_deep_scrub":"260'548","last_deep_scrub_stamp":"2024-09-17T21:50:32.744209+0000","last_clean_scrub_stamp":"2024-09-17T21:50:32.744209+0000","objects_scrubbed":23,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:05:52.252732+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002413152,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":5952,"num_read_kb":32246,"num_write":571,"num_write_kb":18470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"260'2","reported_seq":531,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:39.820685+0000","last_change":"2024-09-17T21:50:39.820745+0000","last_active":"2024-09-17T21:50:39.820685+0000","last_peered":"2024-09-17T21:50:39.820685+0000","last_clean":"2024-09-17T21:50:39.820685+0000","last_became_active":"2024-09-17T21:49:07.910446+0000","last_became_peered":"2024-09-17T21:49:07.910446+0000","last_unstale":"2024-09-17T21:50:39.820685+0000","last_undegraded":"2024-09-17T21:50:39.820685+0000","last_fullsized":"2024-09-17T21:50:39.820685+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'2","last_scrub_stamp":"2024-09-17T21:50:39.820585+0000","last_deep_scrub":"260'2","last_deep_scrub_stamp":"2024-09-17T21:50:39.820585+0000","last_clean_scrub_stamp":"2024-09-17T21:50:39.820585+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:04:35.646018+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00039309399999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"260'119","reported_seq":661,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:36.403783+0000","last_change":"2024-09-17T21:50:36.403824+0000","last_active":"2024-09-17T21:50:36.403783+0000","last_peered":"2024-09-17T21:50:36.403783+0000","last_clean":"2024-09-17T21:50:36.403783+0000","last_became_active":"2024-09-17T21:49:22.710895+0000","last_became_peered":"2024-09-17T21:49:22.710895+0000","last_unstale":"2024-09-17T21:50:36.403783+0000","last_undegraded":"2024-09-17T21:50:36.403783+0000","last_fullsized":"2024-09-17T21:50:36.403783+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'119","last_scrub_stamp":"2024-09-17T21:50:36.403692+0000","last_deep_scrub":"260'119","last_deep_scrub_stamp":"2024-09-17T21:50:36.403692+0000","last_clean_scrub_stamp":"2024-09-17T21:50:36.403692+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:57:42.778515+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"260'514","reported_seq":7509,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.807874+0000","last_change":"2024-09-17T21:50:35.807935+0000","last_active":"2024-09-17T21:50:35.807874+0000","last_peered":"2024-09-17T21:50:35.807874+0000","last_clean":"2024-09-17T21:50:35.807874+0000","last_became_active":"2024-09-17T21:49:11.915056+0000","last_became_peered":"2024-09-17T21:49:11.915056+0000","last_unstale":"2024-09-17T21:50:35.807874+0000","last_undegraded":"2024-09-17T21:50:35.807874+0000","last_fullsized":"2024-09-17T21:50:35.807874+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'514","last_scrub_stamp":"2024-09-17T21:50:35.807783+0000","last_deep_scrub":"260'514","last_deep_scrub_stamp":"2024-09-17T21:50:35.807783+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.807783+0000","objects_scrubbed":22,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T23:44:31.410722+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.001646961,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":7499,"num_read_kb":58807,"num_write":531,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":5466866,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"260'869","reported_seq":11228,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.286212+0000","last_change":"2024-09-17T21:50:35.286259+0000","last_active":"2024-09-17T21:50:35.286212+0000","last_peered":"2024-09-17T21:50:35.286212+0000","last_clean":"2024-09-17T21:50:35.286212+0000","last_became_active":"2024-09-17T21:49:10.917135+0000","last_became_peered":"2024-09-17T21:49:10.917135+0000","last_unstale":"2024-09-17T21:50:35.286212+0000","last_undegraded":"2024-09-17T21:50:35.286212+0000","last_fullsized":"2024-09-17T21:50:35.286212+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'869","last_scrub_stamp":"2024-09-17T21:50:35.286117+0000","last_deep_scrub":"260'869","last_deep_scrub_stamp":"2024-09-17T21:50:35.286117+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.286117+0000","objects_scrubbed":17,"log_size":869,"log_dups_size":0,"ondisk_log_size":869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:07:52.167401+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.001673913,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":17,"num_read":15762,"num_read_kb":70321,"num_write":796,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":70976059,"num_keys_recovered":46,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"260'566","reported_seq":5494,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:38.423558+0000","last_change":"2024-09-17T21:50:38.423621+0000","last_active":"2024-09-17T21:50:38.423558+0000","last_peered":"2024-09-17T21:50:38.423558+0000","last_clean":"2024-09-17T21:50:38.423558+0000","last_became_active":"2024-09-17T21:49:22.713321+0000","last_became_peered":"2024-09-17T21:49:22.713321+0000","last_unstale":"2024-09-17T21:50:38.423558+0000","last_undegraded":"2024-09-17T21:50:38.423558+0000","last_fullsized":"2024-09-17T21:50:38.423558+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'566","last_scrub_stamp":"2024-09-17T21:50:38.423465+0000","last_deep_scrub":"260'566","last_deep_scrub_stamp":"2024-09-17T21:50:38.423465+0000","last_clean_scrub_stamp":"2024-09-17T21:50:38.423465+0000","objects_scrubbed":19,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:35:16.293568+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0024270479999999998,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":18,"num_read":4927,"num_read_kb":14309,"num_write":571,"num_write_kb":2542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":2928005,"num_keys_recovered":215,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"260'610","reported_seq":11126,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:35.844747+0000","last_change":"2024-09-17T21:50:35.844808+0000","last_active":"2024-09-17T21:50:35.844747+0000","last_peered":"2024-09-17T21:50:35.844747+0000","last_clean":"2024-09-17T21:50:35.844747+0000","last_became_active":"2024-09-17T21:48:10.690096+0000","last_became_peered":"2024-09-17T21:48:10.690096+0000","last_unstale":"2024-09-17T21:50:35.844747+0000","last_undegraded":"2024-09-17T21:50:35.844747+0000","last_fullsized":"2024-09-17T21:50:35.844747+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'610","last_scrub_stamp":"2024-09-17T21:50:35.844646+0000","last_deep_scrub":"260'610","last_deep_scrub_stamp":"2024-09-17T21:50:35.844646+0000","last_clean_scrub_stamp":"2024-09-17T21:50:35.844646+0000","objects_scrubbed":21,"log_size":610,"log_dups_size":0,"ondisk_log_size":610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T21:51:38.699284+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0.00079950599999999996,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":10975,"num_read_kb":58277,"num_write":542,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":36865856,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"260'642","reported_seq":19444,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:37.861092+0000","last_change":"2024-09-17T21:50:37.861177+0000","last_active":"2024-09-17T21:50:37.861092+0000","last_peered":"2024-09-17T21:50:37.861092+0000","last_clean":"2024-09-17T21:50:37.861092+0000","last_became_active":"2024-09-17T21:48:07.676590+0000","last_became_peered":"2024-09-17T21:48:07.676590+0000","last_unstale":"2024-09-17T21:50:37.861092+0000","last_undegraded":"2024-09-17T21:50:37.861092+0000","last_fullsized":"2024-09-17T21:50:37.861092+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'642","last_scrub_stamp":"2024-09-17T21:50:37.860991+0000","last_deep_scrub":"260'642","last_deep_scrub_stamp":"2024-09-17T21:50:37.860991+0000","last_clean_scrub_stamp":"2024-09-17T21:50:37.860991+0000","objects_scrubbed":21,"log_size":642,"log_dups_size":0,"ondisk_log_size":642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:55:11.375101+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.0029012579999999999,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":18257,"num_read_kb":77078,"num_write":607,"num_write_kb":16869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":104,"num_bytes_recovered":539841,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"260'900","reported_seq":10303,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:39.435003+0000","last_change":"2024-09-17T21:50:39.435056+0000","last_active":"2024-09-17T21:50:39.435003+0000","last_peered":"2024-09-17T21:50:39.435003+0000","last_clean":"2024-09-17T21:50:39.435003+0000","last_became_active":"2024-09-17T21:49:22.713256+0000","last_became_peered":"2024-09-17T21:49:22.713256+0000","last_unstale":"2024-09-17T21:50:39.435003+0000","last_undegraded":"2024-09-17T21:50:39.435003+0000","last_fullsized":"2024-09-17T21:50:39.435003+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'900","last_scrub_stamp":"2024-09-17T21:50:39.434893+0000","last_deep_scrub":"260'900","last_deep_scrub_stamp":"2024-09-17T21:50:39.434893+0000","last_clean_scrub_stamp":"2024-09-17T21:50:39.434893+0000","objects_scrubbed":14,"log_size":900,"log_dups_size":0,"ondisk_log_size":900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:46:34.688093+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.0041644220000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":11958,"num_read_kb":74544,"num_write":957,"num_write_kb":32191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":528,"num_bytes_recovered":99506614,"num_keys_recovered":92,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"260'463","reported_seq":6199,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:34.840212+0000","last_change":"2024-09-17T21:50:34.840243+0000","last_active":"2024-09-17T21:50:34.840212+0000","last_peered":"2024-09-17T21:50:34.840212+0000","last_clean":"2024-09-17T21:50:34.840212+0000","last_became_active":"2024-09-17T21:47:07.873073+0000","last_became_peered":"2024-09-17T21:47:07.873073+0000","last_unstale":"2024-09-17T21:50:34.840212+0000","last_undegraded":"2024-09-17T21:50:34.840212+0000","last_fullsized":"2024-09-17T21:50:34.840212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'463","last_scrub_stamp":"2024-09-17T21:50:34.840122+0000","last_deep_scrub":"260'463","last_deep_scrub_stamp":"2024-09-17T21:50:34.840122+0000","last_clean_scrub_stamp":"2024-09-17T21:50:34.840122+0000","objects_scrubbed":26,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T03:17:12.919186+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0.0030578519999999998,"stat_sum":{"num_bytes":0,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":26,"num_read":5627,"num_read_kb":34429,"num_write":597,"num_write_kb":25941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":1195556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"260'1051","reported_seq":17788,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:41.847839+0000","last_change":"2024-09-17T21:50:41.847897+0000","last_active":"2024-09-17T21:50:41.847839+0000","last_peered":"2024-09-17T21:50:41.847839+0000","last_clean":"2024-09-17T21:50:41.847839+0000","last_became_active":"2024-09-17T21:49:22.713242+0000","last_became_peered":"2024-09-17T21:49:22.713242+0000","last_unstale":"2024-09-17T21:50:41.847839+0000","last_undegraded":"2024-09-17T21:50:41.847839+0000","last_fullsized":"2024-09-17T21:50:41.847839+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'1051","last_scrub_stamp":"2024-09-17T21:50:41.847747+0000","last_deep_scrub":"260'1051","last_deep_scrub_stamp":"2024-09-17T21:50:41.847747+0000","last_clean_scrub_stamp":"2024-09-17T21:50:41.847747+0000","objects_scrubbed":22,"log_size":1051,"log_dups_size":0,"ondisk_log_size":1051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T22:10:44.268705+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.0024169450000000002,"stat_sum":{"num_bytes":0,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":22,"num_read":19495,"num_read_kb":66374,"num_write":1359,"num_write_kb":30239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":90173557,"num_keys_recovered":96,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"260'644","reported_seq":7493,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:38.870696+0000","last_change":"2024-09-17T21:50:38.870742+0000","last_active":"2024-09-17T21:50:38.870696+0000","last_peered":"2024-09-17T21:50:38.870696+0000","last_clean":"2024-09-17T21:50:38.870696+0000","last_became_active":"2024-09-17T21:49:01.882368+0000","last_became_peered":"2024-09-17T21:49:01.882368+0000","last_unstale":"2024-09-17T21:50:38.870696+0000","last_undegraded":"2024-09-17T21:50:38.870696+0000","last_fullsized":"2024-09-17T21:50:38.870696+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'644","last_scrub_stamp":"2024-09-17T21:50:38.870610+0000","last_deep_scrub":"260'644","last_deep_scrub_stamp":"2024-09-17T21:50:38.870610+0000","last_clean_scrub_stamp":"2024-09-17T21:50:38.870610+0000","objects_scrubbed":21,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:07:47.927305+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00097513899999999998,"stat_sum":{"num_bytes":0,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":21,"num_read":8690,"num_read_kb":24602,"num_write":642,"num_write_kb":5435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":8478338,"num_keys_recovered":44,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"260'577","reported_seq":6486,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:41.431575+0000","last_change":"2024-09-17T21:50:41.431669+0000","last_active":"2024-09-17T21:50:41.431575+0000","last_peered":"2024-09-17T21:50:41.431575+0000","last_clean":"2024-09-17T21:50:41.431575+0000","last_became_active":"2024-09-17T21:49:23.714709+0000","last_became_peered":"2024-09-17T21:49:23.714709+0000","last_unstale":"2024-09-17T21:50:41.431575+0000","last_undegraded":"2024-09-17T21:50:41.431575+0000","last_fullsized":"2024-09-17T21:50:41.431575+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'577","last_scrub_stamp":"2024-09-17T21:50:41.431468+0000","last_deep_scrub":"260'577","last_deep_scrub_stamp":"2024-09-17T21:50:41.431468+0000","last_clean_scrub_stamp":"2024-09-17T21:50:41.431468+0000","objects_scrubbed":23,"log_size":577,"log_dups_size":0,"ondisk_log_size":577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:39:09.481367+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0.001363242,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":4772,"num_read_kb":61222,"num_write":585,"num_write_kb":31941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":640,"num_bytes_recovered":142913984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"260'413","reported_seq":12792,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:39.774076+0000","last_change":"2024-09-17T21:50:39.774127+0000","last_active":"2024-09-17T21:50:39.774076+0000","last_peered":"2024-09-17T21:50:39.774076+0000","last_clean":"2024-09-17T21:50:39.774076+0000","last_became_active":"2024-09-17T21:49:53.677806+0000","last_became_peered":"2024-09-17T21:49:53.677806+0000","last_unstale":"2024-09-17T21:50:39.774076+0000","last_undegraded":"2024-09-17T21:50:39.774076+0000","last_fullsized":"2024-09-17T21:50:39.774076+0000","mapping_epoch":230,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":231,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'413","last_scrub_stamp":"2024-09-17T21:50:39.773988+0000","last_deep_scrub":"260'413","last_deep_scrub_stamp":"2024-09-17T21:50:39.773988+0000","last_clean_scrub_stamp":"2024-09-17T21:50:39.773988+0000","objects_scrubbed":23,"log_size":413,"log_dups_size":0,"ondisk_log_size":413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:39:52.313817+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0.0022036780000000001,"stat_sum":{"num_bytes":0,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":23,"num_read":10169,"num_read_kb":51990,"num_write":466,"num_write_kb":24484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":57538580,"num_keys_recovered":48,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"260'506","reported_seq":10500,"reported_epoch":260,"state":"active+clean","last_fresh":"2024-09-17T21:50:37.446748+0000","last_change":"2024-09-17T21:50:37.446829+0000","last_active":"2024-09-17T21:50:37.446748+0000","last_peered":"2024-09-17T21:50:37.446748+0000","last_clean":"2024-09-17T21:50:37.446748+0000","last_became_active":"2024-09-17T21:49:22.713182+0000","last_became_peered":"2024-09-17T21:49:22.713182+0000","last_unstale":"2024-09-17T21:50:37.446748+0000","last_undegraded":"2024-09-17T21:50:37.446748+0000","last_fullsized":"2024-09-17T21:50:37.446748+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"260'506","last_scrub_stamp":"2024-09-17T21:50:37.446660+0000","last_deep_scrub":"260'506","last_deep_scrub_stamp":"2024-09-17T21:50:37.446660+0000","last_clean_scrub_stamp":"2024-09-17T21:50:37.446660+0000","objects_scrubbed":24,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:29.335875+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0.0013789130000000001,"stat_sum":{"num_bytes":0,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":24,"num_read":8770,"num_read_kb":42462,"num_write":577,"num_write_kb":12397,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":27560597,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":712,"num_object_clones":0,"num_object_copies":1424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":712,"num_whiteouts":708,"num_read":382073,"num_read_kb":1876893,"num_write":22910,"num_write_kb":602361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11204,"num_bytes_recovered":1257863971,"num_keys_recovered":2095,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},"log_size":22069,"ondisk_log_size":22069,"up":64,"acting":64,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":4722944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":5,"up_from":12,"seq":51539607618,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64784,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307056,"statfs":{"total":96636764160,"available":96570425344,"internally_reserved":0,"allocated":3145728,"data_stored":1330161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":148,"seq":635655159839,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63184,"kb_used_data":3648,"kb_used_omap":28,"kb_used_meta":59491,"kb_avail":94308656,"statfs":{"total":96636764160,"available":96572063744,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29616,"internal_metadata":60918864},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":12,"seq":51539607617,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56460,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315380,"statfs":{"total":96636764160,"available":96578949120,"internally_reserved":0,"allocated":3141632,"data_stored":1330142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":12,"seq":51539607617,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48720,"kb_used_data":3648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94323120,"statfs":{"total":96636764160,"available":96586874880,"internally_reserved":0,"allocated":3735552,"data_stored":1920510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":12,"seq":51539607617,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70352,"kb_used_data":3072,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94301488,"statfs":{"total":96636764160,"available":96564723712,"internally_reserved":0,"allocated":3145728,"data_stored":1330161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640322,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60628,"kb_used_data":3076,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311212,"statfs":{"total":96636764160,"available":96574681088,"internally_reserved":0,"allocated":3149824,"data_stored":1330180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15744,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T21:50:54.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T21:50:54.503 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T21:50:54.503 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T21:50:54.503 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T21:50:54.503 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.541 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T21:50:54.541 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T21:50:54.541 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.616 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T21:50:54.616 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T21:50:54.616 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.700 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T21:50:54.700 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T21:50:54.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.743 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T21:50:54.743 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T21:50:54.743 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.794 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T21:50:54.794 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T21:50:54.794 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.843 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T21:50:54.844 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T21:50:54.844 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T21:50:54.844 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.873 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T21:50:54.874 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T21:50:54.874 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T21:50:54.874 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.886 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T21:50:54.886 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T21:50:54.886 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.938 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T21:50:54.938 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T21:50:54.938 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T21:50:54.952 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T21:50:54.952 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T21:50:54.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | head -n 1 2024-09-17T21:50:55.016 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:50:55.017 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T21:50:55.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:50:55.257 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T21:50:55.364 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:50:55.364 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T21:50:55.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:50:55.423 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T21:50:55.616 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:50:55.617 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T21:50:55.691 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:50:55.691 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T21:50:55.747 INFO:tasks.ceph:Archiving mon data... 2024-09-17T21:50:55.748 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/data/mon.a.tgz 2024-09-17T21:50:55.748 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-17T21:50:55.765 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.cPpDMPaTV4 2024-09-17T21:50:55.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.cPpDMPaTV4 2024-09-17T21:50:55.939 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.cPpDMPaTV4 2024-09-17T21:50:56.024 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.cPpDMPaTV4 is 440KB 2024-09-17T21:50:56.121 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.cPpDMPaTV4 2024-09-17T21:50:56.137 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/data/mon.c.tgz 2024-09-17T21:50:56.137 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-17T21:50:56.191 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.4lVkoVU9Sp 2024-09-17T21:50:56.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4lVkoVU9Sp 2024-09-17T21:50:56.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.4lVkoVU9Sp 2024-09-17T21:50:56.471 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.4lVkoVU9Sp is 577KB 2024-09-17T21:50:56.542 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.4lVkoVU9Sp 2024-09-17T21:50:56.557 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/data/mon.b.tgz 2024-09-17T21:50:56.558 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-17T21:50:56.576 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.vHlcN0Re9v 2024-09-17T21:50:56.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vHlcN0Re9v 2024-09-17T21:50:56.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.vHlcN0Re9v 2024-09-17T21:50:56.883 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.vHlcN0Re9v is 577KB 2024-09-17T21:50:56.953 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.vHlcN0Re9v 2024-09-17T21:50:56.969 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T21:50:56.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:50:56.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:50:57.011 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T21:50:57.060 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:50:57.089 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:50:57.107 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T21:50:57.107 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T21:50:57.108 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi007/crash 2024-09-17T21:50:57.109 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:50:57.136 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi053/crash 2024-09-17T21:50:57.137 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:50:57.162 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi165/crash 2024-09-17T21:50:57.163 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T21:50:57.273 INFO:tasks.ceph:Compressing logs... 2024-09-17T21:50:57.273 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:50:57.277 DEBUG:teuthology.orchestra.run.smithi053:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:50:57.279 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T21:50:57.301 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T21:50:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55380.log 2024-09-17T21:50:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40639.log 2024-09-17T21:50:57.302 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64672.log 2024-09-17T21:50:57.302 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55380.log.gz 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32139.log 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43868.log 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32099.log 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32139.log.gz 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41161.log 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.43868.log: /var/log/ceph/ceph-client.admin.32099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32099.log.gz 2024-09-17T21:50:57.319 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.41161.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32222.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32222.log.gz 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32019.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32179.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32019.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32019.log.gz 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-client.admin.32179.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32179.log.gz 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42050.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42939.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.42050.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/objectstore_tool.42939.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T21:50:57.320 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mgr.x.log: 84.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-mon.b.log: 94.9% -- replaced with /var/log/ceph/objectstore_tool.43868.log.gz 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.41161.log.gz 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.42939.log.gz 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.42050.log.gz 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32059.log 2024-09-17T21:50:57.321 INFO:teuthology.orchestra.run.smithi053.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.32059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32059.log.gz 2024-09-17T21:50:57.413 INFO:teuthology.orchestra.run.smithi053.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29850.log 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26995.log 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28839.log 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29850.log.gz 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.26995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29120.log 2024-09-17T21:50:57.452 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.26995.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30912.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28839.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28376.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.29120.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29719.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.30912.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30585.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28376.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30691.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30585.log: 96.2% -- replaced with /var/log/ceph/ceph-client.admin.29719.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29744.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30585.log.gz 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30630.log 2024-09-17T21:50:57.453 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30691.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28175.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29744.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29366.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 94.4% -- replaced with /var/log/ceph/ceph-client.admin.30630.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27579.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.28175.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30411.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.29366.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27545.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.27579.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28105.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30411.log.gz 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29540.log 2024-09-17T21:50:57.454 INFO:teuthology.orchestra.run.smithi165.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.27545.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27789.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.28105.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27404.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.29540.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29147.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 76.1% -- replaced with /var/log/ceph/ceph-client.admin.27789.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27264.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.27404.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29531.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29147.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28200.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.admin.27264.log.gz 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29321.log 2024-09-17T21:50:57.455 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29531.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27676.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28200.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29321.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29920.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27676.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27325.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29920.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29884.log 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27325.log.gz 2024-09-17T21:50:57.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27850.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 53.2% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29296.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29884.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27499.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27850.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29287.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.29296.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27640.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27499.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28455.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29287.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27289.log 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27640.log.gz 2024-09-17T21:50:57.457 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27649.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.admin.28455.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28419.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27289.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27289.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30210.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 76.2% -- replaced with /var/log/ceph/ceph-client.admin.27649.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27359.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 56.4% -- replaced with /var/log/ceph/ceph-client.admin.28419.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27999.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.30210.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27474.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27359.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29789.log 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr: 57.3% -- replaced with /var/log/ceph/ceph-client.admin.27999.log.gz 2024-09-17T21:50:57.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29436.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.27474.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28035.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29789.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30796.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 81.9% -- replaced with /var/log/ceph/ceph-client.admin.29436.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29260.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 95.2% -- replaced with /var/log/ceph/ceph-client.admin.28035.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30796.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27746.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 52.9% -- replaced with /var/log/ceph/ceph-client.admin.29260.log.gz 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28480.log 2024-09-17T21:50:57.459 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28550.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27746.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30734.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28480.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27929.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28550.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 83.1% -- replaced with /var/log/ceph/ceph-client.admin.30734.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29156.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.27929.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29181.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27255.log 2024-09-17T21:50:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29929.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 96.0% -- replaced with /var/log/ceph/ceph-client.admin.29156.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29181.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28516.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27255.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27859.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.29929.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28690.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28516.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29639.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 56.6% -- replaced with /var/log/ceph/ceph-client.admin.27859.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29251.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.28690.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28209.log 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29639.log.gz 2024-09-17T21:50:57.461 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29814.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29251.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27114.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.28209.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28830.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29814.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30420.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27114.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30655.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28830.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30842.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 53.6% -- replaced with /var/log/ceph/ceph-client.admin.30420.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28936.log 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30655.log.gz 2024-09-17T21:50:57.462 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29040.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.admin.30842.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28735.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28936.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27606.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29040.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30131.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.28735.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28970.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27606.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27615.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30131.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30244.log 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28970.log.gz 2024-09-17T21:50:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28726.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.27615.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27685.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 55.4% -- replaced with /var/log/ceph/ceph-client.admin.30244.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27088.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28726.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28586.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 52.1% -- replaced with /var/log/ceph/ceph-client.admin.27685.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30454.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27088.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.27088.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28796.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28586.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28769.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 54.5% -- replaced with /var/log/ceph/ceph-client.admin.30454.log.gz 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30515.log 2024-09-17T21:50:57.464 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28796.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29567.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.28769.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.26953.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30515.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27438.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.29567.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.26953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30490.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.26953.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28279.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 53.0% -- replaced with /var/log/ceph/ceph-client.admin.27438.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27368.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 53.9% -- replaced with /var/log/ceph/ceph-client.admin.30490.log.gz 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30762.log 2024-09-17T21:50:57.465 INFO:teuthology.orchestra.run.smithi165.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.28279.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27298.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.27368.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28270.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30762.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29990.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 57.1% -- replaced with /var/log/ceph/ceph-client.admin.27298.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28559.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28270.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30384.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29990.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28446.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.28559.log.gz 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28349.log 2024-09-17T21:50:57.466 INFO:teuthology.orchestra.run.smithi165.stderr: 54.3% -- replaced with /var/log/ceph/ceph-client.admin.30384.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30140.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28446.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29710.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 24.7% -- replaced with /var/log/ceph/ceph-client.admin.28349.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29190.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.30140.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.29710.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27157.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29710.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27185.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.29190.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28866.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 52.7% -- replaced with /var/log/ceph/ceph-client.admin.27157.log.gz 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30833.log 2024-09-17T21:50:57.467 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27185.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27825.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28866.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28909.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 17.6% -- replaced with /var/log/ceph/ceph-client.admin.30833.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30903.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.27825.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29893.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 55.9% -- replaced with /var/log/ceph/ceph-client.admin.28909.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30280.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30903.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27886.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.29893.log.gz 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29780.log 2024-09-17T21:50:57.468 INFO:teuthology.orchestra.run.smithi165.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.30280.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30524.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27886.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30445.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29780.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28656.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 52.4% -- replaced with /var/log/ceph/ceph-client.admin.30524.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27228.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30445.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.28656.log: -- /var/log/ceph/ceph-client.admin.27719.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28656.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27920.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 55.7% -- replaced with /var/log/ceph/ceph-client.admin.27228.log.gz 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27895.log 2024-09-17T21:50:57.469 INFO:teuthology.orchestra.run.smithi165.stderr: 26.5% -- replaced with /var/log/ceph/ceph-client.admin.27719.log.gz 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29461.log 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27920.log.gz 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28410.log 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.27895.log.gz 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27194.log 2024-09-17T21:50:57.470 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29461.log.gz 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27123.log 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28410.log.gz 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29497.log 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr: 53.3% -- replaced with /var/log/ceph/ceph-client.admin.27194.log.gz 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27148.log 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.27123.log.gz 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30375.log 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29497.log.gz 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27710.log 2024-09-17T21:50:57.472 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27148.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28096.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30375.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29086.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27710.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27780.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28096.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28060.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29086.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27780.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.28060.log: /var/log/ceph/ceph-client.admin.27395.log 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28060.log.gz 2024-09-17T21:50:57.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30341.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27334.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27395.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28489.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30341.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29999.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28489.log: 94.7% -- replaced with /var/log/ceph/ceph-client.admin.27334.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30165.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 25.3% -- replaced with /var/log/ceph/ceph-client.admin.28489.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.29999.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30095.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 95.2%/var/log/ceph/ceph-client.admin.30165.log: -- replaced with /var/log/ceph/ceph-client.admin.29999.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28945.log 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30165.log.gz 2024-09-17T21:50:57.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28760.log 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30095.log.gz 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27429.log 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28760.log: 95.4% -- replaced with /var/log/ceph/ceph-client.admin.28945.log.gz 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27990.log 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28760.log.gz 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29111.log 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27429.log.gz 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28315.log 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.27990.log.gz 2024-09-17T21:50:57.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30481.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29111.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30314.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.28315.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30805.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.30481.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29753.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 52.5% -- replaced with /var/log/ceph/ceph-client.admin.30314.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30350.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.30805.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28166.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29753.log: 54.0% -- replaced with /var/log/ceph/ceph-client.admin.29753.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28620.log 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.30350.log.gz 2024-09-17T21:50:57.476 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30235.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28166.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30070.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28620.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30594.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30235.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30551.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 52.0% -- replaced with /var/log/ceph/ceph-client.admin.30070.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30271.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 55.0% -- replaced with /var/log/ceph/ceph-client.admin.30594.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27465.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30551.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27219.log 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30271.log.gz 2024-09-17T21:50:57.477 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27508.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27465.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29217.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.27219.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27079.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.27508.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30305.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29217.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28069.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27079.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29963.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30305.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30867.log 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr: 56.3% -- replaced with /var/log/ceph/ceph-client.admin.28069.log.gz 2024-09-17T21:50:57.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29226.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 55.1% -- replaced with /var/log/ceph/ceph-client.admin.29963.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28665.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 20.0% -- replaced with /var/log/ceph/ceph-client.admin.30867.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29648.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29226.log: 53.5% -- replaced with /var/log/ceph/ceph-client.admin.29226.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28875.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.admin.28665.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28979.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.29648.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29330.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 54.6% -- replaced with /var/log/ceph/ceph-client.admin.28875.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30560.log 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr: 56.9% -- replaced with /var/log/ceph/ceph-client.admin.28979.log.gz 2024-09-17T21:50:57.479 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29506.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 35.3% -- replaced with /var/log/ceph/ceph-client.admin.29330.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30725.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 77.4% -- replaced with /var/log/ceph/ceph-client.admin.30560.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28525.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.29506.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28340.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.30725.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28130.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28340.log: 94.9% -- replaced with /var/log/ceph/ceph-client.admin.28525.log.gzgzip 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27965.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28340.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28805.log 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28130.log.gz 2024-09-17T21:50:57.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30024.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.27965.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28139.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.28805.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29400.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.30024.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27755.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 55.3% -- replaced with /var/log/ceph/ceph-client.admin.28139.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30201.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 56.8% -- replaced with /var/log/ceph/ceph-client.admin.29400.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30876.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.27755.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.30201.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.30771.log 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.30201.log.gz 2024-09-17T21:50:57.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29006.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 53.8% -- replaced with /var/log/ceph/ceph-client.admin.30876.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28385.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 78.7% -- replaced with /var/log/ceph/ceph-client.admin.30771.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28026.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29006.log: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29006.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29576.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.admin.28385.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28595.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28026.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28629.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.29576.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30664.log 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28595.log.gz 2024-09-17T21:50:57.482 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30104.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 77.9% -- replaced with /var/log/ceph/ceph-client.admin.28629.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29954.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.30664.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27570.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 55.5% -- replaced with /var/log/ceph/ceph-client.admin.30104.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28900.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.29954.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28236.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27570.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27536.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28900.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28699.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.28236.log.gz 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.27536.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29823.log 2024-09-17T21:50:57.483 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27536.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28245.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 54.2% -- replaced with /var/log/ceph/ceph-client.admin.28699.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29610.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 74.2% -- replaced with /var/log/ceph/ceph-client.admin.29823.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30061.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28245.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28306.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.29610.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27956.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30061.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.28306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.28306.log.gz 2024-09-17T21:50:57.484 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30621.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.27956.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29015.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 54.9% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27816.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.30621.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29682.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 58.3% -- replaced with /var/log/ceph/ceph-client.admin.29015.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.27816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30174.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 18.8% -- replaced with /var/log/ceph/ceph-client.admin.27816.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29427.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.29682.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26986.log 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr: 52.2% -- replaced with /var/log/ceph/ceph-client.admin.30174.log.gz 2024-09-17T21:50:57.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29601.log 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr: 18.2% -- replaced with /var/log/ceph/ceph-client.admin.29427.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.26986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29357.log 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.26986.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29859.log 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29601.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30700.log 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr: 19.4% -- replaced with /var/log/ceph/ceph-client.admin.29357.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29859.log: 94.5% -- replaced with /var/log/ceph/ceph-client.admin.29859.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.30700.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.30700.log.gz 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m0.079s 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m0.078s 2024-09-17T21:50:57.486 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.119s 2024-09-17T21:50:57.542 INFO:teuthology.orchestra.run.smithi053.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40639.log.gz 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64672.log.gz 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53428.log 2024-09-17T21:50:57.554 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54460.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67468.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53428.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35612.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54460.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39333.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67468.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39979.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35612.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35388.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39333.log.gz 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2024-09-17T21:50:57.555 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39979.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40183.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35388.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67997.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65089.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40183.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57634.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67997.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40803.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65089.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64162.log 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57634.log.gz 2024-09-17T21:50:57.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48034.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40803.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67445.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64162.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62984.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48034.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42293.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67445.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62984.log.gzgzip 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62963.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42293.log: /var/log/ceph/ceph-client.admin.45641.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42293.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66779.log 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62963.log.gz 2024-09-17T21:50:57.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66737.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45641.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35256.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66779.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53154.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66737.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58686.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35256.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67661.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53154.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66211.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58686.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43940.log 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67661.log.gz 2024-09-17T21:50:57.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36522.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66211.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46469.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43940.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68418.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36522.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67913.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46469.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67093.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68418.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45777.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67913.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67093.log.gz 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53639.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37159.log 2024-09-17T21:50:57.559 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45777.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67167.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53639.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43255.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37159.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67167.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log/var/log/ceph/ceph-client.admin.43255.log: 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36158.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27380.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36158.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.36158.log.gz /var/log/ceph/ceph-client.admin.59978.log 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T21:50:57.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.27380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68207.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27380.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53963.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59978.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45150.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68207.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68207.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53963.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45150.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45150.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35609.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55452.log 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35609.log.gz 2024-09-17T21:50:57.561 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60431.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41078.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55452.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60431.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60431.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.63416.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41078.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63416.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65732.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36018.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65732.log.gz 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37241.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.36018.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60472.log 2024-09-17T21:50:57.562 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36018.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37241.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52547.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60472.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66275.log 0.0% 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40721.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52547.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66275.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58853.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40721.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58853.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58853.log.gz 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57029.log 2024-09-17T21:50:57.563 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65331.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57029.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57029.log.gz -5 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.65525.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48218.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65331.log.gz 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65525.log.gzgzip 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61591.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61027.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48218.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48218.log.gz 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62743.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61591.log.gz 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64930.log 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61027.log.gz 2024-09-17T21:50:57.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62743.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50334.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62628.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64930.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50334.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38374.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58614.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62628.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38374.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38374.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.66569.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57428.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58614.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66569.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.66569.log.gz 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38634.log 2024-09-17T21:50:57.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57428.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38634.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56973.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56973.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65092.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42346.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66485.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65092.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55910.log 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42346.log.gz 2024-09-17T21:50:57.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66485.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39121.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38749.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55910.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39121.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39121.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.59212.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65586.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38749.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67405.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59212.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36966.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65586.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67405.log.gz 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2024-09-17T21:50:57.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36966.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gzgzip 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54593.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63987.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54593.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32115.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40380.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63987.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32115.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.32115.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.64284.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56260.log 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40380.log.gz 2024-09-17T21:50:57.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64284.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37454.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36079.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56260.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37454.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37454.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39398.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36079.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62132.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68333.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39398.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66611.log 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62132.log.gz 2024-09-17T21:50:57.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66921.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68333.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66611.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.66611.log.gz --verbose 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.38077.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66919.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66921.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66921.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38077.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41742.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63538.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66919.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41742.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45980.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57272.log 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63538.log.gz 2024-09-17T21:50:57.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45980.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45980.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44051.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31808.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57272.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44051.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.44051.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37944.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44988.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31808.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35614.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37944.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46234.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44988.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58687.log 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35614.log.gz 2024-09-17T21:50:57.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35430.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46234.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58687.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40366.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49243.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35430.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40366.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.40366.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47164.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36312.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49243.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47164.log.gz 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68039.log 2024-09-17T21:50:57.572 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36312.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44645.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68039.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.68039.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42071.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66317.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44645.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42071.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50630.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66317.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50630.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36438.log 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-09-17T21:50:57.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64559.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36438.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36438.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61980.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64559.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54694.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41528.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61980.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35607.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54694.log.gz 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46818.log 2024-09-17T21:50:57.574 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41528.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52254.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35607.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59427.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46818.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65289.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52254.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66143.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59427.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63291.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65289.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43685.log 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66143.log.gz 2024-09-17T21:50:57.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63291.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50863.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43685.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59744.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61007.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50863.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65690.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59744.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47801.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61007.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52000.log 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65690.log.gz 2024-09-17T21:50:57.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54848.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47801.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67829.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52000.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55930.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54848.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67829.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48076.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55930.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66191.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48076.log.gz 2024-09-17T21:50:57.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58916.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66191.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58392.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58749.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58916.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38161.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58392.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67745.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58749.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66653.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38161.log.gz 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44828.log 2024-09-17T21:50:57.578 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67745.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68081.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66653.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61192.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44828.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62696.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68081.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45231.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61192.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62696.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66916.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45231.log.gz 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-09-17T21:50:57.579 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59480.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66916.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67113.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61295.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59480.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67113.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50974.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61295.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.42938.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56617.log 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-09-17T21:50:57.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53680.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50974.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42018.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56617.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53680.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36704.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42018.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63793.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48298.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36704.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67577.log 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63793.log.gz 2024-09-17T21:50:57.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48298.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48298.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35613.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67577.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58339.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35613.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35613.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40444.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68123.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58339.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35886.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40444.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62215.log 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68123.log.gz 2024-09-17T21:50:57.582 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49032.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35886.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52902.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62215.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67347.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49032.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57808.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52902.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53912.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67347.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45580.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57808.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59500.log 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53912.log.gz 2024-09-17T21:50:57.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60832.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45580.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59500.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51753.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60832.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 0.0% 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51753.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52376.log 2024-09-17T21:50:57.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45939.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62529.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52376.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47963.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45939.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62529.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42979.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67871.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47963.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42979.log.gz 2024-09-17T21:50:57.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36707.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67871.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66048.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39886.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36707.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48826.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66048.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44768.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37802.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38649.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48826.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68165.log 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44768.log.gz 2024-09-17T21:50:57.586 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65411.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38649.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54928.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68165.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46536.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65411.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49607.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54928.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54928.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66917.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46536.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50335.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49607.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61725.log 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66917.log.gz 2024-09-17T21:50:57.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68291.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50335.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35885.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61725.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55125.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68291.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36038.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35885.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55125.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55125.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58791.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36038.log.gz 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41701.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32037.log 2024-09-17T21:50:57.588 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58791.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48908.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41701.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.32037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62629.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32037.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46214.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48908.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57386.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62629.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66914.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46214.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59152.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57386.log.gz 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57346.log 2024-09-17T21:50:57.589 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66914.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48652.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59152.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54185.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57346.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51309.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48652.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42568.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54185.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68458.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51309.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42568.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42568.log.gz 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51895.log 2024-09-17T21:50:57.590 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68458.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60566.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51895.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51895.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49283.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60566.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67703.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49283.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67703.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63331.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51035.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63331.log.gz 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2024-09-17T21:50:57.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51035.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54278.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61275.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54278.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57151.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61275.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56382.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57151.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57151.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67955.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56382.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60092.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67955.log.gz 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67096.log 2024-09-17T21:50:57.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60092.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44492.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67096.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51752.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40844.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53306.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61818.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51752.log: gzip/var/log/ceph/ceph-client.admin.53306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51752.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53306.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.42488.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61818.log: -- /var/log/ceph/ceph-client.admin.45538.log 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61818.log.gz 2024-09-17T21:50:57.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42488.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67404.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45538.log: /var/log/ceph/ceph-client.admin.67404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66443.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67404.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45538.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60112.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66443.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62488.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66443.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47710.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60112.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49627.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62488.log.gz 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-09-17T21:50:57.594 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47710.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49627.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49627.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: 91.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50701.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44903.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52102.log: gzip/var/log/ceph/ceph-client.admin.50701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50701.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.36154.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58102.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44903.log: /var/log/ceph/ceph-client.admin.36154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56891.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44903.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36154.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58102.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57747.log 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58102.log.gz 2024-09-17T21:50:57.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56891.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56891.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44665.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57747.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57747.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52650.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44665.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63374.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52650.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40185.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63374.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40185.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40505.log 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2024-09-17T21:50:57.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68375.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40505.log: -- /var/log/ceph/ceph-client.admin.59797.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40505.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68375.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62448.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59797.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.35337.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59797.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62448.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62448.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35337.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36156.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35337.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.36156.log: -- /var/log/ceph/ceph-client.admin.39334.log 2024-09-17T21:50:57.597 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36156.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55258.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43339.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39334.log: /var/log/ceph/ceph-client.admin.55258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35760.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55258.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39334.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60852.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35760.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60852.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67787.log 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-09-17T21:50:57.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55696.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67787.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63692.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55696.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48724.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.63692.log: -- /var/log/ceph/ceph-client.admin.40043.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63692.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48724.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48724.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56146.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40043.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40043.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65966.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56146.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56146.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51958.log 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65966.log.gz 2024-09-17T21:50:57.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56932.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66695.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51958.log: /var/log/ceph/ceph-client.admin.56932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41426.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56932.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51958.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66695.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58042.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41426.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42663.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58042.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52567.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42663.log.gz 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58022.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52567.log: -- /var/log/ceph/ceph-client.admin.67174.log 2024-09-17T21:50:57.600 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52567.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58022.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67174.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.35606.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67174.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67325.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39611.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35606.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45345.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35606.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51662.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39611.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45345.log: /var/log/ceph/ceph-client.admin.66359.log 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45345.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51662.log.gz 2024-09-17T21:50:57.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54995.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66359.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60349.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41313.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54995.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35472.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60349.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41313.log: /var/log/ceph/ceph-client.admin.56065.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41313.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35472.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61130.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65091.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56065.log: /var/log/ceph/ceph-client.admin.61130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50295.log 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56065.log.gz 2024-09-17T21:50:57.602 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61130.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65091.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66401.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50295.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.67619.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50295.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64357.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66401.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61470.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51835.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64357.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67619.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64357.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68249.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61470.log.gz 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60770.log 2024-09-17T21:50:57.603 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51835.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68249.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60254.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60770.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60770.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39959.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60254.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36270.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39959.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64888.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36270.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37709.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64888.log.gz 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58998.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67172.log 2024-09-17T21:50:57.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37709.log: /var/log/ceph/ceph-client.admin.58998.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.37709.log.gz --verbose 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.65772.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58998.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57471.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67172.log: /var/log/ceph/ceph-client.admin.65772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42724.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67172.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65772.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57471.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47205.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42724.log: gzip -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42724.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.62569.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47205.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47205.log.gz 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35312.log 2024-09-17T21:50:57.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62569.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62569.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63967.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35312.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35312.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66102.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63967.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66527.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66102.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.43572.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66102.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66920.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66527.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43572.log: /var/log/ceph/ceph-client.admin.52967.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43572.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.66920.log: /var/log/ceph/ceph-client.admin.49831.log 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66920.log.gz 2024-09-17T21:50:57.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52967.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52967.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44946.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49831.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63218.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44946.log: /var/log/ceph/ceph-client.admin.64814.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44946.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63218.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35825.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63218.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60542.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64814.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34951.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35793.log 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35825.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60542.log.gz 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T21:50:57.607 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50103.log 2024-09-17T21:50:57.608 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34951.log.gz 2024-09-17T21:50:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35793.log.gz 2024-09-17T21:50:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50103.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50103.log.gz 2024-09-17T21:50:57.608 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T21:50:57.742 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T21:51:01.820 INFO:teuthology.orchestra.run.smithi053.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T21:51:02.040 INFO:teuthology.orchestra.run.smithi007.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T21:51:04.113 INFO:teuthology.orchestra.run.smithi053.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T21:51:04.954 INFO:teuthology.orchestra.run.smithi007.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T21:51:06.153 INFO:teuthology.orchestra.run.smithi007.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T21:51:06.155 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-17T21:51:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m8.865s 2024-09-17T21:51:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m21.035s 2024-09-17T21:51:06.155 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m1.014s 2024-09-17T21:51:08.438 INFO:teuthology.orchestra.run.smithi053.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T21:51:08.440 INFO:teuthology.orchestra.run.smithi053.stderr: 2024-09-17T21:51:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:real 0m11.148s 2024-09-17T21:51:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:user 0m22.017s 2024-09-17T21:51:08.440 INFO:teuthology.orchestra.run.smithi053.stderr:sys 0m0.791s 2024-09-17T21:51:08.441 INFO:tasks.ceph:Archiving logs... 2024-09-17T21:51:08.441 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi007/log 2024-09-17T21:51:08.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:51:11.056 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi053/log 2024-09-17T21:51:11.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:51:13.524 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi165/log 2024-09-17T21:51:13.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T21:51:13.722 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T21:51:13.736 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T21:51:13.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:51:13.739 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:51:13.741 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T21:51:13.835 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T21:51:13.844 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T21:51:13.844 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:51:13.846 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:51:13.849 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T21:51:13.862 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:51:13.864 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:51:13.866 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -490us[ -504us] +/- 44ms 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -345us[ -345us] +/- 75ms 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 +1098us[+1084us] +/- 67ms 2024-09-17T21:51:13.867 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:51:13.869 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:51:13.869 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-17T21:51:13.869 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -477us[ -478us] +/- 44ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 42 -335us[ -335us] +/- 75ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +1117us[+1116us] +/- 67ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -469us[ -450us] +/- 44ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 -346us[ -346us] +/- 75ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +1117us[+1135us] +/- 67ms 2024-09-17T21:51:13.870 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T21:51:13.872 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T21:51:13.880 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T21:51:13.880 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T21:51:13.895 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T21:51:14.004 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T21:51:14.090 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T21:51:14.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:51:14.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:14.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:51:14.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:14.222 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-17T21:51:14.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:14.253 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T21:51:14.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T21:51:14.324 INFO:teuthology.task.internal:Duration was 732.473653 seconds 2024-09-17T21:51:14.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T21:51:14.333 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T21:51:14.334 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:51:14.336 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:51:14.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T21:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:51:14.380 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:51:14.383 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T21:51:14.868 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T21:51:14.868 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-17T21:51:14.868 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:51:14.894 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-17T21:51:14.895 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:51:14.919 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-09-17T21:51:14.919 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T21:51:14.944 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T21:51:14.944 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:51:14.946 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:51:14.962 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T21:51:15.013 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T21:51:15.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:51:15.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:51:15.033 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T21:51:15.164 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T21:51:15.173 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T21:51:15.174 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:51:15.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:51:15.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T21:51:15.250 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T21:51:15.259 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.261 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.267 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.283 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-17T21:51:15.289 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-17T21:51:15.312 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-09-17T21:51:15.337 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:15.366 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:15.381 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T21:51:15.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T21:51:15.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T21:51:15.404 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T21:51:15.405 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi007 2024-09-17T21:51:15.405 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:51:15.445 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi053 2024-09-17T21:51:15.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:51:15.480 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_21:16:02-rbd-squid-distro-default-smithi/7909503/remote/smithi165 2024-09-17T21:51:15.480 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T21:51:15.518 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T21:51:15.519 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:51:15.521 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:51:15.523 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T21:51:15.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T21:51:15.577 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T21:51:15.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T21:51:15.585 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T21:51:15.585 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:51:15.588 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:51:15.590 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T21:51:15.601 INFO:teuthology.orchestra.run.smithi007.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:51 /home/ubuntu/cephtest 2024-09-17T21:51:15.603 INFO:teuthology.orchestra.run.smithi053.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:51 /home/ubuntu/cephtest 2024-09-17T21:51:15.621 INFO:teuthology.orchestra.run.smithi165.stdout: 265881 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 21:51 /home/ubuntu/cephtest 2024-09-17T21:51:15.622 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T21:51:15.631 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T21:51:15.658 INFO:teuthology.run:Summary data: description: rbd/device/{base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-bitmap supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nbd} duration: 732.4736528396606 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-17T21:51:15.658 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T21:51:15.739 INFO:teuthology.run:pass